Multiagent Reinforcement Learning with Spiking and Non-Spiking Agents in the Iterated Prisoner's Dilemma

Authors:
Vassilis Vassiliades;Aristodemos Cleanthous;Chris Christodoulou
Affiliations:
Department of Computer Science, University of Cyprus, Nicosia, Cyprus 1678;Department of Computer Science, University of Cyprus, Nicosia, Cyprus 1678;Department of Computer Science, University of Cyprus, Nicosia, Cyprus 1678
Venue:
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part I
Year:
2009

Citing 20
Cited 0

Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching

Machine Learning
The dynamics of reinforcement learning in cooperative multiagent systems

AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Gradient descent for general reinforcement learning

Proceedings of the 1998 conference on Advances in neural information processing systems II
Learning to Predict by the Methods of Temporal Differences

Machine Learning
Convergent Gradient Ascent in General-Sum Games

ECML '02 Proceedings of the 13th European Conference on Machine Learning
Friend-or-Foe Q-learning in General-Sum Games

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Nash q-learning for general-sum stochastic games

The Journal of Machine Learning Research
Reinforcement Learning of Coordination in Heterogeneous Cooperative Multi-Agent Systems

AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 3
A general criterion and an algorithmic framework for learning in multi-agent systems

Machine Learning
AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents

Machine Learning
Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity

Neural Computation
If multi-agent learning is the answer, what is the question?

Artificial Intelligence
An economist's perspective on multi-agent learning

Artificial Intelligence
Agendas for multi-agent learning

Artificial Intelligence
Multiagent learning is not the answer. It is the question

Artificial Intelligence
What evolutionary game theory tells us about multiagent learning

Artificial Intelligence
A hierarchy of prescriptive goals for multiagent learning

Artificial Intelligence
Multi-agent learning and the descriptive value of simple models

Artificial Intelligence
A spiking neural network model of an actor-critic learning agent

Neural Computation
Rational and convergent learning in stochastic games

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper investigates Multiagent Reinforcement Learning (MARL) in a general-sum game where the payoffs' structure is such that the agents are required to exploit each other in a way that benefits all agents. The contradictory nature of these games makes their study in multiagent systems quite challenging. In particular, we investigate MARL with spiking and non-spiking agents in the Iterated Prisoner's Dilemma by exploring the conditions required to enhance its cooperative outcome. According to the results, this is enhanced by: (i) a mixture of positive and negative payoff values and a high discount factor in the case of non-spiking agents and (ii) having longer eligibility trace time constant in the case of spiking agents. Moreover, it is shown that spiking and non-spiking agents have similar behaviour and therefore they can equally well be used in any multiagent interaction setting. For training the spiking agents, a novel and necessary modification enhances competition to an existing learning rule based on stochastic synaptic transmission.