Technical Note: \cal Q-Learning
Machine Learning
Tree based discretization for continuous state space reinforcement learning
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
A Tutorial on Support Vector Machines for Pattern Recognition
Data Mining and Knowledge Discovery
The Complexity of Decentralized Control of Markov Decision Processes
Mathematics of Operations Research
TTree: Tree-Based State Generalization with Temporally Abstract Actions
Proceedings of the 5th International Symposium on Abstraction, Reformulation and Approximation
The complexity of multiagent systems: the price of silence
AAMAS '03 Proceedings of the second international joint conference on Autonomous agents and multiagent systems
Tree-Based Batch Mode Reinforcement Learning
The Journal of Machine Learning Research
Derivation and Analysis of Basic Computational Operations of Thalamocortical Circuits
Journal of Cognitive Neuroscience
Engines of the brain: the computational instruction set of human cognition
AI Magazine - Special issue on achieving human-level AI through integrated systems and research
UAI'97 Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence
Nearest neighbor pattern classification
IEEE Transactions on Information Theory
| Hi-index | 0.00 |
A primary challenge of agent-based reinforcement learning in complex and uncertain environments is escalating computational complexity with the number of the states. Hierarchical, or tree-based, state representation provides a promising approach to complexity reduction through clustering and sequencing of similar states. We introduce the Q-Tree algorithm to utilize the data history of state transition information to automatically construct such a representation and to obtain a series of linear separations between state clusters to facilitate learning. Empirical results for the canonical PuddleWorld problem are provided to validate the proposed algorithm; extensions of the PuddleWorld problem obtained by adding random noise dimensions are solved by the Q-Tree algorithm, while traditional tabular Q-learning cannot accommodate random state elements within the same number of learning trials. The results show that the Q-Tree algorithm can reject state dimensions that do not aid learning by analyzing weights of all linear classifiers for a hierarchical state representation.