Analysis of Augmented Krylov Subspace Methods
SIAM Journal on Matrix Analysis and Applications
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Neuro-Dynamic Programming
Iterative Methods for Sparse Linear Systems
Iterative Methods for Sparse Linear Systems
Fast direct policy evaluation using multiscale analysis of Markov diffusion processes
ICML '06 Proceedings of the 23rd international conference on Machine learning
Learning representation and control in continuous Markov decision processes
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Samuel meets Amarel: automating value function approximation using global state space analysis
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Constructing basis functions from directed graphs for value function approximation
Proceedings of the 24th international conference on Machine learning
Proceedings of the 25th international conference on Machine learning
Learning Representation and Control in Markov Decision Processes: New Frontiers
Foundations and Trends® in Machine Learning
Compact spectral bases for value function approximation using Kronecker factorization
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
Fast spectral learning using Lanczos eigenspace projections
AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 3
Character animation in two-player adversarial games
ACM Transactions on Graphics (TOG)
Basis function construction for hierarchical reinforcement learning
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Automatic induction of bellman-error features for probabilistic planning
Journal of Artificial Intelligence Research
Basis function discovery using spectral clustering and bisimulation metrics
ALA'11 Proceedings of the 11th international conference on Adaptive and Learning Agents
An online kernel-based clustering approach for value function approximation
SETN'12 Proceedings of the 7th Hellenic conference on Artificial Intelligence: theories and applications
Construction of approximation spaces for reinforcement learning
The Journal of Machine Learning Research
Hi-index | 0.00 |
Recently, a method based on Laplacian eigenfunctions was proposed to automatically construct a basis for value function approximation in MDPs. We show that its success may be explained by drawing a connection between the spectrum of the Laplacian and the value function of the MDP. This explanation helps us to identify more precisely the conditions that this method requires to achieve good performance. Based on this, we propose a modification of the Laplacian method for which we derive an analytical bound on the approximation error. Further, we show that the method is related the augmented Krylov methods, commonly used to solve sparse linear systems. Finally, we empirically demonstrate that in basis construction the augmented Krylov methods may significantly outperform the Laplacian methods in terms of both speed and quality.