Principles of artificial intelligence
Principles of artificial intelligence
A model for reasoning about persistence and causation
Computational Intelligence
Modeling a dynamic and uncertain world I: symbolic and probabilistic reasoning about change
Artificial Intelligence
Learning to act using real-time dynamic programming
Artificial Intelligence - Special volume on computational research on interaction and agency, part 1
Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning
Artificial Intelligence
LAO: a heuristic search algorithm that finds solutions with loops
Artificial Intelligence - Special issue on heuristic search in artificial intelligence
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Neuro-Dynamic Programming
State abstraction for programmable reinforcement learning agents
Eighteenth national conference on Artificial intelligence
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
A heuristic search approach to planning with continuous resources in stochastic domains
Journal of Artificial Intelligence Research
Memory-enhanced evolutionary robotics: the echo state network approach
CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
Hi-index | 0.00 |
Many MDPs exhibit an hierarchical structure where the agent needs to perform various subtasks that are coupled only by a small sub-set of variables containing, notably, shared resources. Previous work has shown how this hierarchical structure can be exploited by solving several sub-MDPs representing the different subtasks in different calling contexts, and a root MDP responsible for sequencing and synchronizing the subtasks, instead of a huge MDP representing the whole problem. Another important idea used by efficient algorithms for solving flat MDPs, such as (L)AO and (L)RTDP, is to exploit reachability information and an admissible heuristics in order to accelerate the search by pruning states that cannot be reached from a given starting state under an optimal policy. In this paper, we combine both ideas and develop a variant of the AO* algorithm for performing forward heuristic search in hierarchical models. This algorithm shows great performance improvements over hierarchical approaches using standard MDP solvers such as Value Iteration, as well as with respect to AO applied to a flat representation of the problem. Moreover, it presents a general new method for accelerating AO and other forward search algorithms. Substantial performance gains may be obtained in these algorithms by partitioning the set of search nodes, and solving a subset of nodes completely before propagating the results to other subsets.