The complexity of Markov decision processes
Mathematics of Operations Research
A survey of algorithmic methods for partially observed Markov decision processes
Annals of Operations Research
Reasoning about knowledge
An introduction to Kolmogorov complexity and its applications (2nd ed.)
An introduction to Kolmogorov complexity and its applications (2nd ed.)
Planning and acting in partially observable stochastic domains
Artificial Intelligence
Rational Coordination in Multi-Agent Environments
Autonomous Agents and Multi-Agent Systems
The Complexity of Decentralized Control of Markov Decision Processes
Mathematics of Operations Research
Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Sequential Optimality and Coordination in Multiagent Systems
IJCAI '99 Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence
Artificial Intelligence: A Modern Approach
Artificial Intelligence: A Modern Approach
On the undecidability of probabilistic planning and related stochastic optimization problems
Artificial Intelligence - special issue on planning with uncertainty and incomplete information
Planning and control in stochastic domains with imperfect information
Planning and control in stochastic domains with imperfect information
Value-function approximations for partially observable Markov decision processes
Journal of Artificial Intelligence Research
Taming decentralized POMDPs: towards efficient policy computation for multiagent settings
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Multi-agent influence diagrams for representing and solving games
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Approximating state estimation in multiagent settings using particle filters
Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
Exact solutions of interactive POMDPs using behavioral equivalence
AAMAS '06 Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems
Learning to communicate in a decentralized environment
Autonomous Agents and Multi-Agent Systems
Interactive dynamic influence diagrams
Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Graphical models for online solutions to interactive POMDPs
Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Not all agents are equal: scaling up distributed POMDPs for agent networks
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Value-based observation compression for DEC-POMDPs
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Interaction-driven Markov games for decentralized multiagent planning under uncertainty
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Graphical models for interactive POMDPs: representations and solutions
Autonomous Agents and Multi-Agent Systems
Improved approximation of interactive dynamic influence diagrams using discriminative model updates
Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Towards strategic Kriegspiel play with opponent modeling
Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
On the difficulty of achieving equilibrium in interactive POMDPs
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Point-based dynamic programming for DEC-POMDPs
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
A particle filtering based approach to approximating interactive POMDPs
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Approximate solutions of interactive dynamic influence diagrams using model clustering
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Generalized point based value iteration for interactive POMDPs
AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 1
Optimal and approximate Q-value functions for decentralized POMDPs
Journal of Artificial Intelligence Research
Monte Carlo sampling methods for approximating interactive POMDPs
Journal of Artificial Intelligence Research
Memory-bounded dynamic programming for DEC-POMDPs
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Agent Influence and Intelligent Approximation in Multiagent Problems
WI-IAT '09 Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 02
Speeding up exact solutions of interactive dynamic influence diagrams using action equivalence
IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
A PGM framework for recursive modeling of players in simple sequential Bayesian games
International Journal of Approximate Reasoning
Modeling recursive reasoning by humans using empirically informed interactive POMDPs
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Coordinated learning in multiagent MDPs with infinite state-space
Autonomous Agents and Multi-Agent Systems
Mutual state-based capabilities for role assignment in heterogeneous teams
Proceedings of the 3rd International Symposium on Practical Cognitive Agents and Robots
Decentralized MDPs with sparse interactions
Artificial Intelligence
Using iterated reasoning to predict opponent strategies
The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
A partition-based first-order probabilistic logic to represent interactive beliefs
SUM'11 Proceedings of the 5th international conference on Scalable uncertainty management
Scalable multiagent planning using probabilistic inference
IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
A framework for modeling population strategies by depth of reasoning
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Improved use of partial policies for identifying behavioral equivalence
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Generalized and bounded policy iteration for finitely-nested interactive POMDPs: scaling up
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Higher-order social cognition in rock-paper-scissors: a simulation study
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Modeling deep strategic reasoning by humans in competitive games
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
GaTAC: a scalable and realistic testbed for multiagent decision making (demonstration)
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Exploiting model equivalences for solving interactive dynamic influence diagrams
Journal of Artificial Intelligence Research
Developing a Repeated Multi-agent Constant-Sum Game Algorithm Using Human Computation
WI-IAT '12 Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 02
Learning Communication in Interactive Dynamic Influence Diagrams
WI-IAT '12 Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 02
Bayesian interaction shaping: learning to influence strategic interactions in mixed robotic domains
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Decentralized multi-robot cooperation with auctioned POMDPs
International Journal of Robotics Research
Incremental clustering and expansion for faster optimal planning in decentralized POMDPs
Journal of Artificial Intelligence Research
Undecidability in epistemic planning
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Bimodal switching for online planning in multiagent settings
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Hi-index | 0.01 |
This paper extends the framework of partially observable Markov decision processes (POMDPs) to multi-agent settings by incorporating the notion of agent models into the state space. Agents maintain beliefs over physical states of the environment and over models of other agents, and they use Bayesian updates to maintain their beliefs over time. The solutions map belief states to actions. Models of other agents may include their belief states and are related to agent types considered in games of incomplete information. We express the agents' autonomy by postulating that their models are not directly manipulable or observable by other agents. We show that important properties of POMDPs, such as convergence of value iteration, the rate of convergence, and piece-wise linearity and convexity of the value functions carry over to our framework. Our approach complements a more traditional approach to interactive settings which uses Nash equilibria as a solution paradigm. We seek to avoid some of the drawbacks of equilibria which may be non-unique and do not capture off-equilibrium behaviors. We do so at the cost of having to represent, process and continuously revise models of other agents. Since the agent's beliefs may be arbitrarily nested, the optimal solutions to decision making problems are only asymptotically computable. However, approximate belief updates and approximately optimal plans are computable. We illustrate our framework using a simple application domain, and we show examples of belief updates and value functions.