Using iterated reasoning to predict opponent strategies
The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
| Hi-index | 0.00 |
Multi-agent systems that use traditional game-theoretic analysis for decision making take a normative approach, in which agents' optimal decisions are derived from the game description. This approach is insufficient to model agents which are uncertain about the structure of the game, the strategies of other agents or whether other agents are behaving optimally. This thesis presents a compact, natural and highly expressive language for describing and reasoning about agents' beliefs and decision-making processes, called Networks of Influence Diagrams (NID). NIDs are graphical structures in which agents' mental models are represented as nodes in a network. NIDs are recursive: a mental model for an agent may itself contain models of the mental models of other agents. NIDs define a new kind of solution concept for games that makes a distinction between what agents should do and what they are expected to do. This distinction makes it possible for NIDs to quantify the extent to which an agent deviates from its optimal strategy, which is the sole course of action prescribed by the game-theoretic approaches. This thesis provides methods for solving NIDs that can compute an equilibrium that satisfies the conditions defined by the solution concept. These methods exploit the graph structure of the NID to integrate agents' different beliefs into a single computational framework. A formal comparison between NIDs and classical game-theoretic formalisms is presented, showing that NIDs may be exponentially more compact than formalisms that are traditionally used to model uncertainty in games. Several algorithms are provided that enable NID parameters to be learned from data consisting of agents' interactions. These algorithms are evaluated in two separate domains in which NIDs were used to learn agents' decision-making models. In the first domain, involving other learner agents, a NID agent was able to construct a belief hierarchy over the decision-making models of other agents and outperform them. In the second domain, involving humans, a NID agent was able to learn how social factors affected the negotiation behavior of different types of people in a strategic setting. In this domain the NID agent outperformed other agents that negotiated using game-theoretic equilibrium strategies.