Team-partitioned, opaque-transition reinforcement learning
Proceedings of the third annual conference on Autonomous Agents
Multiagent systems: a modern approach to distributed artificial intelligence
Multiagent systems: a modern approach to distributed artificial intelligence
Elevator Group Control Using Multiple Reinforcement Learning Agents
Machine Learning
Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning
Artificial Intelligence
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Reinforcement Learning in the Multi-Robot Domain
Autonomous Robots
Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Hierarchical Optimization of Policy-Coupled Semi-Markov Decision Processes
ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
Hierarchical control and learning for markov decision processes
Hierarchical control and learning for markov decision processes
Hierarchical reinforcement learning with the MAXQ value function decomposition
Journal of Artificial Intelligence Research
Recent Advances in Hierarchical Reinforcement Learning
Discrete Event Dynamic Systems
State abstraction for programmable reinforcement learning agents
Eighteenth national conference on Artificial intelligence
Recent Advances in Hierarchical Reinforcement Learning
Discrete Event Dynamic Systems
Learning to Communicate and Act Using Hierarchical Reinforcement Learning
AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 3
Cooperative Multi-Agent Learning: The State of the Art
Autonomous Agents and Multi-Agent Systems
Hierarchical Co-evolution of Cooperating Agents Acting in the Brain-Arena
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Efficient multi-agent reinforcement learning through automated supervision
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 3
Hierarchical Classifiers for Complex Spatio-temporal Concepts
Transactions on Rough Sets IX
Reinforcement Learning: A Tutorial Survey and Recent Advances
INFORMS Journal on Computing
Solving multiagent assignment Markov decision processes
Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Integrating organizational control into multi-agent learning
Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
CCDC'09 Proceedings of the 21st annual international conference on Chinese control and decision conference
Transfer learning via relational templates
ILP'09 Proceedings of the 19th international conference on Inductive logic programming
Decentralized MDPs with sparse interactions
Artificial Intelligence
Cognitive policy learner: biasing winning or losing strategies
The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Holonic multi-agent system for traffic signals control
Engineering Applications of Artificial Intelligence
Applying hybrid learning approach to RoboCup's strategy
Journal of Systems and Software
Robotic Urban Search and Rescue: A Survey from the Control Perspective
Journal of Intelligent and Robotic Systems
| Hi-index | 0.00 |
In this paper we investigate the use of hierarchical reinforcement learning to speed up the acquisition of cooperative multi-agent tasks. We extend the MAXQ framework to the multi-agent case. Each agent uses the same MAXQ hierarchy to decompose a task into sub-tasks. Learning is decentralized, with each agent learning three interrelated skills: how to perform subtasks, which order to do them in, and how to coordinate with other agents. Coordination skills among agents are learned by using joint actions at the highest level(s) of the hierarchy. The Q nodes at the highest level(s) of the hierarchy are configured to represent the joint task-action space among multiple agents. In this approach, each agent only knows what other agents are doing at the level of sub-tasks, and is unaware of lower level (primitive) actions. This hierarchical approach allows agents to learn coordination faster by sharing information at the level of sub-tasks, rather than attempting to learn coordination taking into account primitive joint state-action values. We apply this hierarchical multi-agent reinforcement learning algorithm to a complex AGV scheduling task and compare its performance and speed with other learning approaches, including flat multi-agent, single agent using MAXQ, selfish multiple agents using MAXQ (where each agent acts independently without communicating with the other agents), as well as several well-known AGV heuristics like "first come first serve", "highest queue first" and "nearest station first". We also compare the tradeoffs in learning speed vs. performance of modeling joint action values at multiple levels in the MAXQ hierarchy.