The complexity of decentralized control of Markov decision processes

Authors:
Daniel S. Bernstein;Shlomo Zilberstein;Neil Immerman
Affiliations:
Department of Computer Science, University of Massachusetts, Amherst, Massachusetts;Department of Computer Science, University of Massachusetts, Amherst, Massachusetts;Department of Computer Science, University of Massachusetts, Amherst, Massachusetts
Venue:
UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
Year:
2000

Citing 12
Cited 16

The complexity of Markov decision processes

Mathematics of Operations Research
Collaborative plans for complex group action

Artificial Intelligence
Distributed problem solving and planning

Multiagent systems
Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork

Artificial Intelligence - Special issue on Robocop: the first step
On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
An integrated system for multi-rover scientific exploration

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
Reflections on the Nature of Multi-Agent Coordination and Its Implications for an Agent Architecture

Autonomous Agents and Multi-Agent Systems
Learning to Cooperate via Policy Search

UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Multiple-person alternation

SFCS '79 Proceedings of the 20th Annual Symposium on Foundations of Computer Science
Planning and acting in partially observable stochastic domains

Artificial Intelligence
Solving POMDPs by searching in policy space

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Incremental pruning: a simple, fast, exact method for partially observable Markov decision processes

UAI'97 Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence

Introducing Communication in Dis-POMDPs with Locality of Interaction

WI-IAT '08 Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 02
Comparison of tightly and loosely coupled decision paradigms in multiagent expedition

International Journal of Approximate Reasoning
Towards Addressing Model Uncertainty: Robust Execution-Time Coordination for Teamwork

WI-IAT '11 Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 02
Coordinating teams in uncertain environments: a hybrid BDI-POMDP approach

ProMAS'04 Proceedings of the Second international conference on Programming Multi-Agent Systems
An overview of cooperative and competitive multiagent learning

LAMAS'05 Proceedings of the First international conference on Learning and Adaption in Multi-Agent Systems
Flood disaster mitigation: a real-world challenge problem for multi-agent unmanned surface vehicles

AAMAS'11 Proceedings of the 10th international conference on Advanced Agent Technology
Online planning for ad hoc autonomous agent teams

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume One
Decentralised channel allocation and information sharing for teams of cooperative agents

Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Decentralized Bayesian reinforcement learning for online agent collaboration

Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Solving decentralized POMDP problems using genetic algorithms

Autonomous Agents and Multi-Agent Systems
Bayesian interaction shaping: learning to influence strategic interactions in mixed robotic domains

Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Collaborative health care plan support

Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Distributed reasoning for multiagent simple temporal problems

Journal of Artificial Intelligence Research
Agent-based decentralised coordination for sensor networks using the max-sum algorithm

Autonomous Agents and Multi-Agent Systems
Multiagent meta-level control for radar coordination

Web Intelligence and Agent Systems
Map partitioning to approximate an exploration strategy in mobile robotics

Multiagent and Grid Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Planning for distributed agents with partial state information is considered from a decisiontheoretic perspective. We describe generalizations of both the MDP and POMDP models that allow for decentralized control. For even a small number of agents, the finite-horizon problems corresponding to both of our models are complete for nondeterministic exponential time. These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov processes. In contrast to the MDP and POMDP problems, the problems we consider provably do not admit polynomialtime algorithms and most likely require doubly exponential time to solve in the worst case. We have thus provided mathematical evidence corresponding to the intuition that decentralized planning problems cannot easily be reduced to centralized problems and solved exactly using established techniques.