Incremental pruning: a simple, fast, exact method for partially observable Markov decision processes

Authors:
Anthony Cassandra;Michael L. Littman;Nevin L. Zhang
Affiliations:
Computer Science Dept., Brown University, Providence, RI;Dept. of Computer Science, Duke University, Durham, NC;Computer Science Dept., The Hong Kong U. of Sci. & Tech., Kowloon, HK
Venue:
UAI'97 Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence
Year:
1997

Citing 11
Cited 83

A Survey of solution techniques for the partially observed Markov decision process

Annals of Operations Research
Artificial intelligence: a modern approach

Artificial intelligence: a modern approach
Acting optimally in partially observable stochastic domains

AAAI'94 Proceedings of the twelfth national conference on Artificial intelligence (vol. 2)
Cost-effective sensing during plan execution

AAAI'94 Proceedings of the twelfth national conference on Artificial intelligence (vol. 2)
An algorithm for probabilistic planning

Artificial Intelligence - Special volume on planning and scheduling
Incremental Markov-Model Planning

ICTAI '96 Proceedings of the 8th International Conference on Tools with Artificial Intelligence
Efficient dynamic-programming updates in partially observable Markov decision processes

Efficient dynamic-programming updates in partially observable Markov decision processes
Algorithms for partially observable markov decision processes

Algorithms for partially observable markov decision processes
Approximating optimal policies for partially observable stochastic domains

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
Computing optimal policies for partially observable decision processes using compact representations

AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 2
Reinforcement learning with perceptual aliasing: the perceptual distinctions approach

AAAI'92 Proceedings of the tenth national conference on Artificial intelligence

Integrating POMDP and reinforcement learning for a two layer simulated robot architecture

Proceedings of the third annual conference on Autonomous Agents
Complexity of finite-horizon Markov decision process problems

Journal of the ACM (JACM)
The Complexity of Decentralized Control of Markov Decision Processes

Mathematics of Operations Research
Value Iteration over Belief Subspace

ECSQARU '01 Proceedings of the 6th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Space-Progressive Value Iteration: An Anytime Algorithm for a Class of POMDPs

ECSQARU '01 Proceedings of the 6th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Hidden-Mode Markov Decision Processes for Nonstationary Sequential Decision Making

Sequence Learning - Paradigms, Algorithms, and Applications
Logic, Knowledge Representation, and Bayesian Decision Theory

CL '00 Proceedings of the First International Conference on Computational Logic
The size of MDP factored policies

Eighteenth national conference on Artificial intelligence
Contingent planning under uncertainty via stochastic satisfiability

Artificial Intelligence - special issue on planning with uncertainty and incomplete information
Equivalence notions and model minimization in Markov decision processes

Artificial Intelligence - special issue on planning with uncertainty and incomplete information
Learning low dimensional predictive representations

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Region-based incremental pruning for POMDPs

UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Dynamic programming for structured continuous Markov decision problems

UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Heuristic search value iteration for POMDPs

UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Spoken dialogue management using probabilistic reasoning

ACL '00 Proceedings of the 38th Annual Meeting on Association for Computational Linguistics
An integrated token-based algorithm for scalable coordination

Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
Exploiting belief bounds: practical POMDPs for personal assistant agents

Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
Heuristic anytime approaches to stochastic decision processes

Journal of Heuristics
Winning back the CUP for distributed POMDPs: planning over continuous belief spaces

AAMAS '06 Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems
Point-Based Value Iteration for Continuous POMDPs

The Journal of Machine Learning Research
Partially observable Markov decision processes with imprecise parameters

Artificial Intelligence
Sequential Monte Carlo in reachability heuristics for probabilistic planning

Artificial Intelligence
A Near Optimal Policy for Channel Allocation in Cognitive Radio

Recent Advances in Reinforcement Learning
Partially Observed Markov Decision Process Multiarmed Bandits---Structural Results

Mathematics of Operations Research
Stochastic local search for POMDP controllers

AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Dynamic programming for partially observable stochastic games

AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Compact, convex upper bound iteration for approximate POMDP planning

AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Efficient maximization in solving POMDPs

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Planning in models that combine memory with predictive representations of state

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Rover science autonomy: probabilistic planning for science-aware exploration doctoral consortium thesis summary

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 4
Point-based policy iteration

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Scaling up: solving POMDPs through value based clustering

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Symbolic heuristic search value iteration for factored POMDPs

AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 2
Value-function approximations for partially observable Markov decision processes

Journal of Artificial Intelligence Research
Speeding up the convergence of value iteration in partially observable Markov decision processes

Journal of Artificial Intelligence Research
On polynomial sized MDP succinct policies

Journal of Artificial Intelligence Research
Restricted value iteration: theory and algorithms

Journal of Artificial Intelligence Research
Hybrid BDI-POMDP framework for multiagent teaming

Journal of Artificial Intelligence Research
Perseus: randomized point-based value iteration for POMDPs

Journal of Artificial Intelligence Research
Anytime point-based approximations for large POMDPs

Journal of Artificial Intelligence Research
Online planning algorithms for POMDPs

Journal of Artificial Intelligence Research
Policy iteration for decentralized control of Markov decision processes

Journal of Artificial Intelligence Research
A model approximation scheme for planning in partially observable stochastic domains

Journal of Artificial Intelligence Research
AEMS: an anytime online search algorithm for approximate policy refinement in large POMDPs

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Towards efficient computation of error bounded solutions in POMDPs: expected value approximation and dynamic disjunctive beliefs

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Taming decentralized POMDPs: towards efficient policy computation for multiagent settings

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
An improved grid-based approximation algorithm for POMDPs

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 1
Bounded policy iteration for decentralized POMDPs

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Planning and acting in partially observable stochastic domains

Artificial Intelligence
Distributed spectrum sensing and access in cognitive radio networks with energy constraint

IEEE Transactions on Signal Processing
Topological order planner for POMDPs

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Partially Observable Markov Decision Processes: A Geometric Technique and Analysis

Operations Research
Point-based planning for predictive state representations

Canadian AI'08 Proceedings of the Canadian Society for computational studies of intelligence, 21st conference on Advances in artificial intelligence
Evaluating point-based POMDP solvers on multicore machines

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics - Special issue on gait analysis
Dynamic Allocation of Pharmaceutical Detailing and Sampling for Long-Term Profitability

Marketing Science
Efficient planning in large POMDPs through policy graph based factorized approximations

ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III
A new graphical recursive pruning method for the incremental pruning algorithm

MICAI'10 Proceedings of the 9th Mexican international conference on Advances in artificial intelligence: Part I
A new pruning method for incremental pruning algorithm using a sweeping scan-line through the belief space

MICAI'10 Proceedings of the 9th Mexican international conference on Advances in artificial intelligence: Part I
POMDP filter: pruning POMDP value functions with the Kaczmarz iterative method

MICAI'10 Proceedings of the 9th Mexican international conference on Advances in artificial intelligence: Part I
Planning in stochastic domains for multiple agents with individual continuous resource state-spaces

Autonomous Agents and Multi-Agent Systems
Identifying and exploiting weak-information inducing actions in solving POMDPs

The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
My brain is full: when more memory helps

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
A method for speeding up value iteration in partially observable Markov decision processes

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
The complexity of decentralized control of Markov decision processes

UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
Value-directed belief state approximation for POMDPs

UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
Vector-space analysis of belief-state approximation for POMDPs

UAI'01 Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence
Planning and acting under uncertainty: a new model for spoken dialogue systems

UAI'01 Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence
Solving POMDPs by searching in policy space

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Planning with partially observable Markov decision processes: advances in exact solution method

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Prioritizing point-based POMDP solvers

ECML'06 Proceedings of the 17th European conference on Machine Learning
Optimal limited contingency planning

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Using rewards for belief state updates in partially observable markov decision processes

ECML'05 Proceedings of the 16th European conference on Machine Learning
Belief selection in point-based planning algorithms for POMDPs

AI'06 Proceedings of the 19th international conference on Advances in Artificial Intelligence: Canadian Society for Computational Studies of Intelligence
A POMDP model for guiding taxi cruising in a congested urban city

MICAI'11 Proceedings of the 10th Mexican international conference on Advances in Artificial Intelligence - Volume Part I
Implementation techniques for solving POMDPs in personal assistant agents

ProMAS'05 Proceedings of the Third international conference on Programming Multi-Agent Systems
Point-based value iteration for constrained POMDPs

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
The Skyline algorithm for POMDP value function pruning

Annals of Mathematics and Artificial Intelligence
Tree-based pruning for multiagent POMDPs with delayed communication

Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Finding patterns in an unknown graph

AI Communications - The Symposium on Combinatorial Search
Optimization of Prostate Biopsy Referral Decisions

Manufacturing & Service Operations Management
A survey of point-based POMDP solvers

Autonomous Agents and Multi-Agent Systems
Incremental clustering and expansion for faster optimal planning in decentralized POMDPs

Journal of Artificial Intelligence Research
A survey of multi-objective sequential decision-making

Journal of Artificial Intelligence Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Most exact algorithms for general partially observable Markov decision processes (POMDPs) use a form of dynamic programming in which a piecewise-linear and convex representation of one value function is transformed into another. We examine variations of the "incremental pruning" method for solving this problem and compare them to earlier algorithms from theoretical and empirical perspectives. We find that incremental pruning is presently the most efficient exact method for solving POMDPS.