Feature selection for reinforcement learning: evaluating implicit state-reward dependency via conditional mutual information

Authors:
Hirotaka Hachiya;Masashi Sugiyama
Affiliations:
Tokyo Institute of Technology, Tokyo, Japan;Tokyo Institute of Technology, Tokyo, Japan
Venue:
ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part I
Year:
2010

Citing 12
Cited 5

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Machine Learning
Using expectation-maximization for reinforcement learning

Neural Computation
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
An introduction to variable and feature selection

The Journal of Machine Learning Research
Least-squares policy iteration

The Journal of Machine Learning Research
Information Theory, Inference & Learning Algorithms

Information Theory, Inference & Learning Algorithms
Automatic basis function construction for approximate dynamic programming and reinforcement learning

ICML '06 Proceedings of the 23rd international conference on Machine learning
Analyzing feature generation for value-function approximation

Proceedings of the 24th international conference on Machine learning
Supervised feature selection via dependence estimation

Proceedings of the 24th international conference on Machine learning
An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning

Proceedings of the 25th international conference on Machine learning
Automatic Feature Selection for Model-Based Reinforcement Learning in Factored MDPs

ICMLA '09 Proceedings of the 2009 International Conference on Machine Learning and Applications
A Least-squares Approach to Direct Importance Estimation

The Journal of Machine Learning Research

Sample aware embedded feature selection for reinforcement learning

Proceedings of the 14th annual conference on Genetic and evolutionary computation
Multi-Task reinforcement learning: shaping and feature selection

EWRL'11 Proceedings of the 9th European conference on Recent Advances in Reinforcement Learning
APRIL: active preference learning-based reinforcement learning

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II
Phased searching with NEAT in a Time-Scaled Framework: Experiments on a computer-aided detection system for lung nodules

Artificial Intelligence in Medicine
Learning potential functions and their representations for multi-task reinforcement learning

Autonomous Agents and Multi-Agent Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Model-free reinforcement learning (RL) is a machine learning approach to decision making in unknown environments. However, realworld RL tasks often involve high-dimensional state spaces, and then standard RL methods do not perform well. In this paper, we propose a new feature selection framework for coping with high dimensionality. Our proposed framework adopts conditional mutual information between return and state-feature sequences as a feature selection criterion, allowing the evaluation of implicit state-reward dependency. The conditional mutual information is approximated by a least-squares method, which results in a computationally efficient feature selection procedure. The usefulness of the proposed method is demonstrated on grid-world navigation problems.