Automatic skill acquisition in reinforcement learning using graph centrality measures

Authors:
Parham Moradi;Mohammad Ebrahim Shiri;Ali Ajdari Rad;Alireza Khadivi;Martin Hasler
Affiliations:
Department of Computer Science, Faculty of Mathematics & Computer Science, Amirkabir University of Technology, Tehran, Iran;Department of Computer Science, Faculty of Mathematics & Computer Science, Amirkabir University of Technology, Tehran, Iran;LANOS, IC, Ecole Polytechnique Federal de Lausanne, Lausanne, Switzerland;LANOS, IC, Ecole Polytechnique Federal de Lausanne, Lausanne, Switzerland;LANOS, IC, Ecole Polytechnique Federal de Lausanne, Lausanne, Switzerland
Venue:
Intelligent Data Analysis
Year:
2012

Citing 15
Cited 1

Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching

Machine Learning
Learning hierarchical control structures for multiple tasks and changing environments

Proceedings of the fifth international conference on simulation of adaptive behavior on From animals to animats 5
Reinforcement learning with hierarchies of machines

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning

Artificial Intelligence
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Q-Cut - Dynamic Discovery of Sub-goals in Reinforcement Learning

ECML '02 Proceedings of the 13th European Conference on Machine Learning
Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Recent Advances in Hierarchical Reinforcement Learning

Discrete Event Dynamic Systems
Dynamic abstraction in reinforcement learning via clustering

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Identifying useful subgoals in reinforcement learning by local graph partitioning

ICML '05 Proceedings of the 22nd international conference on Machine learning
Autonomous subgoal discovery and hierarchical abstraction for reinforcement learning using Monte Carlo method

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 4
Hierarchical reinforcement learning with the MAXQ value function decomposition

Journal of Artificial Intelligence Research
Approximating betweenness centrality

WAW'07 Proceedings of the 5th international conference on Algorithms and models for the web-graph
Complexity analysis of real-time reinforcement learning

AAAI'93 Proceedings of the eleventh national conference on Artificial intelligence
Learning skills in reinforcement learning using relative novelty

SARA'05 Proceedings of the 6th international conference on Abstraction, Reformulation and Approximation

A random projection approach for estimation of the betweenness centrality measure

Intelligent Data Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

Mechanisms on automatic discovery of macro actions or skills in reinforcement learning methods are mainly focused on subgoal discovery methods. Among the proposed algorithms, those based on graph centrality measures demonstrate a high performance gain. In this paper, we propose a new graph theoretic approach for automatically identifying and evaluating subgoals. Moreover, we propose a method for providing some useful prior knowledge for corresponding policy of developed skills based on two graph centrality measures, namely node connection graph stability and co-betweenness centrality. Investigating some benchmark problems, we show that the proposed approach improves the learning performance of the agent significantly.