Technical Note: \cal Q-Learning
Machine Learning
The design and analysis of a computational model of cooperative coevolution
The design and analysis of a computational model of cooperative coevolution
The dynamics of reinforcement learning in cooperative multiagent systems
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
A Cooperative Coevolutionary Approach to Function Optimization
PPSN III Proceedings of the International Conference on Evolutionary Computation. The Third Conference on Parallel Problem Solving from Nature: Parallel Problem Solving from Nature
Nash Convergence of Gradient Dynamics in General-Sum Games
UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Reinforcement learning of coordination in cooperative multi-agent systems
Eighteenth national conference on Artificial intelligence
A selection-mutation model for q-learning in multi-agent systems
AAMAS '03 Proceedings of the second international joint conference on Autonomous agents and multiagent systems
An analysis of cooperative coevolutionary algorithms
An analysis of cooperative coevolutionary algorithms
Cooperative Multi-Agent Learning: The State of the Art
Autonomous Agents and Multi-Agent Systems
Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents
Evolutionary Computation
The Cooperative Coevolutionary (1+1) EA
Evolutionary Computation
An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games
Autonomous Agents and Multi-Agent Systems
Handling Communication Restrictions and Team Formation in Congestion Games
Autonomous Agents and Multi-Agent Systems
Robustness in cooperative coevolution
Proceedings of the 8th annual conference on Genetic and evolutionary computation
Selecting informative actions improves cooperative multiagent learning
AAMAS '06 Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems
Lenient learners in cooperative multiagent systems
AAMAS '06 Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems
Evolving distributed agents for managing air traffic
Proceedings of the 9th annual conference on Genetic and evolutionary computation
Predicting and preventing coordination problems in cooperative Q-learning systems
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Improving coevolutionary search for optimal multiagent behaviors
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Exploring the explorative advantage of the cooperative coevolutionary (1+1) EA
GECCO'03 Proceedings of the 2003 international conference on Genetic and evolutionary computation: PartI
Efficient non-linear control through neuroevolution
ECML'06 Proceedings of the 17th European conference on Machine Learning
Formalizing Multi-state Learning Dynamics
WI-IAT '08 Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 02
Cooperative coevolution and univariate estimation of distribution algorithms
Proceedings of the tenth ACM SIGEVO workshop on Foundations of genetic algorithms
Learning the IPA market with individual and social rewards
Web Intelligence and Agent Systems
Dynamic analysis of multiagent Q-learning with ε-greedy exploration
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
State-coupled replicator dynamics
Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Evolving policy geometry for scalable multiagent learning
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
The world of independent learners is not markovian
International Journal of Knowledge-based and Intelligent Engineering Systems
Empirical and theoretical support for lenient learning
The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Shaping fitness functions for coevolving cooperative multiagent systems
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Continuous strategy replicator dynamics for multi-agent Q-learning
Autonomous Agents and Multi-Agent Systems
Controlling tensegrity robots through evolution
Proceedings of the 15th annual conference on Genetic and evolutionary computation
Hi-index | 0.01 |
This paper presents the dynamics of multiple learning agents from an evolutionary game theoretic perspective. We provide replicator dynamics models for cooperative coevolutionary algorithms and for traditional multiagent Q-learning, and we extend these differential equations to account for lenient learners: agents that forgive possible mismatched teammate actions that resulted in low rewards. We use these extended formal models to study the convergence guarantees for these algorithms, and also to visualize the basins of attraction to optimal and suboptimal solutions in two benchmark coordination problems. The paper demonstrates that lenience provides learners with more accurate information about the benefits of performing their actions, resulting in higher likelihood of convergence to the globally optimal solution. In addition, the analysis indicates that the choice of learning algorithm has an insignificant impact on the overall performance of multiagent learning algorithms; rather, the performance of these algorithms depends primarily on the level of lenience that the agents exhibit to one another. Finally, the research herein supports the strength and generality of evolutionary game theory as a backbone for multiagent learning.