Approximate Solutions for Partially Observable Stochastic Games with Common Payoffs

  • Authors:

  • Rosemary Emery-Montemerlo;Geoff Gordon;Jeff Schneider;Sebastian Thrun

  • Affiliations:

  • Carnegie Mellon University;Carnegie Mellon University;Carnegie Mellon University;Stanford University

  • Venue:
  • AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 1
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Partially observable decentralized decision making in robot teams is fundamentally different from decision making in fully observable problems. Team members cannot simply apply single-agent solution techniques in parallel. Instead, we must turn to game theoretic frameworks to correctly model the problem. While partially observable stochastic games (POSGs) provide a solution model for decentralized robot teams, this model quickly becomes intractable. We propose an algorithm that approximates POSGs as a series of smaller, related Bayesian games, using heuristics such as QMDP to provide the future discounted value of actions. This algorithm trades off limited look-ahead in uncertainty for computational feasibility, and results in policies that are locally optimal with respect to the selected heuristic. Empirical results are provided for both a simple problem for which the full POSG can also be constructed, as well as more complex, robot-inspired, problems.