Run the GAMUT: A Comprehensive Approach to Evaluating Game-Theoretic Algorithms

Authors:
Eugene Nudelman;Jennifer Wortman;Yoav Shoham;Kevin Leyton-Brown
Affiliations:
Stanford University;Stanford University;Stanford University;University of British Columbia
Venue:
AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 2
Year:
2004

Citing 8
Cited 39

Simplicial variable dimension algorithms for solving the nonlinear complementarity problem on a product of unit simplices using a general labelling

Mathematics of Operations Research
Technical Note: \cal Q-Learning

Machine Learning
Algorithms, games, and the internet

STOC '01 Proceedings of the thirty-third annual ACM symposium on Theory of computing
Learning the Empirical Hardness of Optimization Problems: The Case of Combinatorial Auctions

CP '02 Proceedings of the 8th International Conference on Principles and Practice of Constraint Programming
Multi-agent algorithms for solving graphical games

Eighteenth national conference on Artificial intelligence
A continuation method for Nash equilibria in structured games

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Complexity results about Nash equilibria

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Rational and convergent learning in stochastic games

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2

Using tabu best-response search to find pure strategy nash equilibria in normal form games

Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games

Autonomous Agents and Multi-Agent Systems
Computing the optimal strategy to commit to

EC '06 Proceedings of the 7th ACM conference on Electronic commerce
A technique for reducing normal-form games to compute a Nash equilibrium

AAMAS '06 Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems
Learning against multiple opponents

AAMAS '06 Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems
A general criterion and an algorithmic framework for learning in multi-agent systems

Machine Learning
If multi-agent learning is the answer, what is the question?

Artificial Intelligence
Applying game theory mechanisms in open agent systems with complete information

Autonomous Agents and Multi-Agent Systems
Multiagent learning in adaptive dynamic systems

Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Effective tag mechanisms for evolving coordination

Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Searching for approximate equilibria in empirical games

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 2
Selecting strategies using empirical game models: an experimental analysis of meta-strategies

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 2
Optimistic-Pessimistic Q-Learning Algorithm for Multi-Agent Systems

MATES '08 Proceedings of the 6th German conference on Multiagent System Technologies
Bounded rationality via recursion

Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Multi-Agent Reinforcement Learning Algorithm with Variable Optimistic-Pessimistic Criterion

Proceedings of the 2008 conference on ECAI 2008: 18th European Conference on Artificial Intelligence
Simple search methods for finding a Nash equilibrium

AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Algorithms for rationalizability and CURB sets

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
A generalized strategy eliminability criterion and computational methods for applying it

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Mixed-integer programming methods for finding Nash equilibria

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Approximate strategic reasoning through hierarchical reduction of large symmetric games

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
The impact of network topology on pure Nash equilibria in graphical games

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
A continuation method for Nash equilibria in structured games

Journal of Artificial Intelligence Research
Computing Bayes-Nash Equilibria through Support Enumeration Methods in Bayesian Two-Player Strategic-Form Games

WI-IAT '09 Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 02
Learning against opponents with bounded memory

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Effective learning in the presence of adaptive counterparts

Journal of Algorithms
Learning graphical game models

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Local search techniques for computing equilibria in two-player general-sum strategic-form games

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Algorithms for closed under rational behavior (CURB) sets

Journal of Artificial Intelligence Research
Linear solvers for nonlinear games: using pivoting algorithms to find Nash equilibria in n-player games

ACM SIGecom Exchanges
Solving Stackelberg games with uncertain observability

The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Approximate nash equilibria in bimatrix games

ICCCI'11 Proceedings of the Third international conference on Computational collective intelligence: technologies and applications - Volume Part II
Learning in one-shot strategic form games

ECML'06 Proceedings of the 17th European conference on Machine Learning
An overview of cooperative and competitive multiagent learning

LAMAS'05 Proceedings of the First international conference on Learning and Adaption in Multi-Agent Systems
Computing nash equilibria of action-graph games via support enumeration

WINE'11 Proceedings of the 7th international conference on Internet and Network Economics
New differential evolution selective mutation operator for the nash equilibria problem

ICCCI'12 Proceedings of the 4th international conference on Computational Collective Intelligence: technologies and applications - Volume Part II
Using representative strategies for finding nash equilibria

Proceedings of the 15th annual conference on Genetic and evolutionary computation
Learning equilibria of games via payoff queries

Proceedings of the fourteenth ACM conference on Electronic commerce
On the verification and computation of strong nash equilibrium

Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Differential evolution as a new method of computing nash equilibria

Transactions on Computational Collective Intelligence IX

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present GAMUT^1, a suite of game generators designed for testing game-theoretic algorithms. We explain why such a generator is necessary, offer a way of visualizing relationships between the sets of games supported by GAMUT, and give an overview of GAMUTýs architecture. We highlight the importance of using comprehensive test data by benchmarking existing algorithms. We show surprisingly large variation in algorithm performance across different sets of games for two widely-studied problems: computing Nash equilibria and multiagent learning in repeated games.