Multiagent Reinforcement Learning for Urban Traffic Control Using Coordination Graphs

Authors:
Lior Kuyer;Shimon Whiteson;Bram Bakker;Nikos Vlassis
Affiliations:
Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands 1098 SJ;Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands 1098 SJ;Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands 1098 SJ;Department of Production Engineering and Management, Technical University of Crete, Chania, Greece
Venue:
ECML PKDD '08 Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I
Year:
2008

Citing 9
Cited 3

Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time

Machine Learning
Multi-Agent Reinforcement Leraning for Traffic Light Control

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Coordinated Reinforcement Learning

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Understanding belief propagation and its generalizations

Exploring artificial intelligence in the new millennium
Collaborative Multiagent Reinforcement Learning by Payoff Propagation

The Journal of Machine Learning Research
A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence

A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence
Exploiting causal independence in Bayesian network inference

Journal of Artificial Intelligence Research
Loopy belief propagation as a basis for communication in sensor networks

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Factor graphs and the sum-product algorithm

IEEE Transactions on Information Theory

Reconciling strategic and tactical decision making in agent-oriented simulation of vehicles in urban traffic

Proceedings of the 4th International ICST Conference on Simulation Tools and Techniques
Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework

Engineering Applications of Artificial Intelligence
Modeling of route planning system based on Q value-based dynamic programming with multi-agent reinforcement learning algorithms

Engineering Applications of Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

Since traffic jams are ubiquitous in the modern world, optimizing the behavior of traffic lights for efficient traffic flow is a critically important goal. Though most current traffic lights use simple heuristic protocols, more efficient controllers can be discovered automatically via multiagent reinforcement learning, where each agent controls a single traffic light. However, in previous work on this approach, agents select only locally optimal actions without coordinating their behavior. This paper extends this approach to include explicit coordination between neighboring traffic lights. Coordination is achieved using the max-plus algorithm, which estimates the optimal joint action by sending locally optimized messages among connected agents. This paper presents the first application of max-plus to a large-scale problem and thus verifies its efficacy in realistic settings. It also provides empirical evidence that max-plus performs well on cyclic graphs, though it has been proven to converge only for tree-structured graphs. Furthermore, it provides a new understanding of the properties a traffic network must have for such coordination to be beneficial and shows that max-plus outperforms previous methods on networks that possess those properties.