Point-based dynamic programming for DEC-POMDPs

Authors:
Daniel Szer;François Charpillet
Affiliations:
MAlA Group, INRIA Lorraine, LORIA, Vandreuvre-lès-Nancy, France;MAlA Group, INRIA Lorraine, LORIA, Vandreuvre-lès-Nancy, France
Venue:
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Year:
2006

Citing 13
Cited 15

Computationally feasible bounds for partially observed Markov decision processes

Operations Research
The dynamics of reinforcement learning in cooperative multiagent systems

AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
The Complexity of Decentralized Control of Markov Decision Processes

Mathematics of Operations Research
Learning to Cooperate via Policy Search

UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Approximate Solutions for Partially Observable Stochastic Games with Common Payoffs

AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 1
Winning back the CUP for distributed POMDPs: planning over continuous belief spaces

AAMAS '06 Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems
Dynamic programming for partially observable stochastic games

AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Solving transition independent decentralized Markov decision processes

Journal of Artificial Intelligence Research
A framework for sequential planning in multi-agent settings

Journal of Artificial Intelligence Research
Taming decentralized POMDPs: towards efficient policy computation for multiagent settings

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Point-based value iteration: an anytime algorithm for POMDPs

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Bounded policy iteration for decentralized POMDPs

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
An optimal best-first search algorithm for solving infinite horizon DEC-POMDPs

ECML'05 Proceedings of the 16th European conference on Machine Learning

Solving Decentralized Continuous Markov Decision Problems with Structured Reward

KI '07 Proceedings of the 30th annual German conference on Advances in Artificial Intelligence
Point-based incremental pruning heuristic for solving finite-horizon DEC-POMDPs

Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Generalized point based value iteration for interactive POMDPs

AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 1
Optimal and approximate Q-value functions for decentralized POMDPs

Journal of Artificial Intelligence Research
Policy iteration for decentralized control of Markov decision processes

Journal of Artificial Intelligence Research
Memory-bounded dynamic programming for DEC-POMDPs

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Heuristic search for identical payoff Bayesian games

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Point-based policy generation for decentralized POMDPs

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
An investigation into mathematical programming for finite horizon decentralized POMDPs

Journal of Artificial Intelligence Research
Point-based bounded policy iteration for decentralized POMDPs

PRICAI'10 Proceedings of the 11th Pacific Rim international conference on Trends in artificial intelligence
Online planning for multi-agent systems with bounded communication

Artificial Intelligence
Planning in stochastic domains for multiple agents with individual continuous resource state-spaces

Autonomous Agents and Multi-Agent Systems
Toward error-bounded algorithms for infinite-horizon DEC-POMDPs

The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Solving decentralized POMDP problems using genetic algorithms

Autonomous Agents and Multi-Agent Systems
Automated generation of interaction graphs for value-factored dec-POMDPs

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

We introduce point-based dynamic programming (DP) for decentralized partially observable Markov decision processes (DEC-POMDPs), a new discrete DP algorithm for planning strategies for cooperative multi-agent systems. Our approach makes a connection between optimal DP algorithms for partially observable stochastic games, and point-based approximations for single-agent POMDPs. We show for the first time how relevant multi-agent belief states can be computed. Building on this insight, we then show how the linear programming part in current multi-agent DP algorithms can be avoided, and how multi-agent DP can thus be applied to solve larger problems. We derive both an optimal and an approximated version of our algorithm, and we show its efficiency on test examples from the literature.