Technical Note: \cal Q-Learning
Machine Learning
Learning hierarchical control structures for multiple tasks and changing environments
Proceedings of the fifth international conference on simulation of adaptive behavior on From animals to animats 5
Reinforcement learning with hierarchies of machines
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning
Artificial Intelligence
Data mining: concepts and techniques
Data mining: concepts and techniques
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Recent Advances in Hierarchical Reinforcement Learning
Discrete Event Dynamic Systems
Q-Cut - Dynamic Discovery of Sub-goals in Reinforcement Learning
ECML '02 Proceedings of the 13th European Conference on Machine Learning
PolicyBlocks: An Algorithm for Creating Useful Macro-Actions in Reinforcement Learning
ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Autonomous discovery of temporal abstractions from interaction with an environment
Autonomous discovery of temporal abstractions from interaction with an environment
Dynamic abstraction in reinforcement learning via clustering
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Discovering hierarchy in reinforcement learning
Discovering hierarchy in reinforcement learning
Identifying useful subgoals in reinforcement learning by local graph partitioning
ICML '05 Proceedings of the 22nd international conference on Machine learning
A causal approach to hierarchical decomposition in reinforcement learning
A causal approach to hierarchical decomposition in reinforcement learning
Reinforcement learning for problems with symmetrical restricted states
Robotics and Autonomous Systems
Hierarchical reinforcement learning with the MAXQ value function decomposition
Journal of Artificial Intelligence Research
Reinforcement learning: a survey
Journal of Artificial Intelligence Research
Review: Data mining techniques and applications - A decade review from 2000 to 2011
Expert Systems with Applications: An International Journal
| Hi-index | 0.00 |
In this paper, we used data mining techniques for the automatic discovering of useful temporal abstraction in reinforcement learning. This idea was motivated by the ability of data mining algorithms in automatic discovering of structures and patterns, when applied to large data sets. The state transitions and action trajectories of the learning agent are stored as the data sets for data mining techniques. The proposed state clustering algorithms partition the state space to different regions. Policies for reaching different parts of the space are separately learned and added to the model in a form of options (macro-actions). The main idea of the proposed action sequence mining is to search for patterns that occur frequently within an agent's accumulated experience. The mined action sequences are also added to the model in a form of options. Our experiments with different data sets indicate a significant speedup of the Q-learning algorithm using the options discovered by the state clustering and action sequence mining algorithms.