Learning to act using real-time dynamic programming
Artificial Intelligence - Special volume on computational research on interaction and agency, part 1
Neuro-Dynamic Programming
Symbolic heuristic search for factored Markov decision processes
Eighteenth national conference on Artificial intelligence
Dynamic Programming
Exploiting first-order regression in inductive policy selection
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Automatic basis function construction for approximate dynamic programming and reinforcement learning
ICML '06 Proceedings of the 23rd international conference on Machine learning
Probabilistic planning via determinization in hindsight
AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 2
The FF planning system: fast plan generation through heuristic search
Journal of Artificial Intelligence Research
Efficient solution algorithms for factored MDPs
Journal of Artificial Intelligence Research
mGPT: a probabilistic planner based on heuristic search
Journal of Artificial Intelligence Research
Generalizing plans to new environments in relational MDPs
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Planning with continuous resources in stochastic domains
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
SPUDD: stochastic planning using decision diagrams
UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Topological value iteration algorithms
Journal of Artificial Intelligence Research
Monitoring the execution of partial-order plans via regression
IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
Towards scalable mdp algorithms
IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
Discovering hidden structure in factored MDPs
Artificial Intelligence
Hi-index | 0.00 |
Past approaches for solving MDPs have several weaknesses: 1) Decision-theoretic computation over the state space can yield optimal results but scales poorly. 2) Value-function approximation typically requires human-specified basis functions and has not been shown successful on nominal ("discrete") domains such as those in the ICAPS planning competitions. 3) Replanning by applying a classical planner to a determinized domain model can generate approximate policies for very large problems but has trouble handling probabilistic subtlety [Little and Thiebaux, 2007]. This paper presents RETRASE, a novel MDP solver, which combines decision theory, function approximation and classical planning in a new way. RETRASE uses classical planning to create basis functions for value-function approximation and applies expected-utility analysis to this compact space. Our algorithm is memory-efficient and fast (due to its compact, approximate representation), returns high-quality solutions (due to the decision-theoretic framework) and does not require additional knowledge from domain engineers (since we apply classical planning to automatically construct the basis functions). Experiments demonstrate that RETRASE outperforms winners from the past three probabilistic-planning competitions on many hard problems.