Multi-agent reinforcement learning: independent vs. cooperative agents
Readings in agents
Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning
Artificial Intelligence
Reinforcement Learning
Sparse Distributed Memory
Brains, Behavior and Robotics
Multiagent Systems: A Survey from a Machine Learning Perspective
Autonomous Robots
Variable Resolution Discretization in Optimal Control
Machine Learning
Fuzzy Model-Based Reinforcement Learning
Advances in Computational Intelligence and Learning: Methods and Applications
Randomized Pursuit-Evasion in Graphs
ICALP '02 Proceedings of the 29th International Colloquium on Automata, Languages and Programming
Randomized Pursuit-Evasion with Local Visibility
SIAM Journal on Discrete Mathematics
Automatic basis function construction for approximate dynamic programming and reinforcement learning
ICML '06 Proceedings of the 23rd international conference on Machine learning
Adaptive Kanerva-based function approximation for multi-agent systems
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 3
Fuzzy Kanerva-based function approximation for reinforcement learning
Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
| Hi-index | 0.00 |
Reinforcement learning has difficulties in solving multi-agent problems because of the inefficiency of function approximation. Sparse distributed memories, which is implemented using Radial Basis Functions or Kanerva Coding, can be used to improve the efficiency. But this approach still often give poor performance when applied to large-scale multi-agent systems. In this paper, we attempt to solve a collection of instances in the predator-prey pursuit domain and argue that the poor performance that we observe is caused by frequent prototype collisions. We show that dynamic prototype allocation and adaptation can give better results by reducing these collisions. We then describe our novel approach, fuzzy Kanerva-based function approximation, that uses a fine-grained fuzzy membership grade to describe a state-action pair's adjacency with respect to each prototype. This approach completely eliminates prototype collisions. We further show that prototype density varies widely across the state-action space and that this variation causes prototypes' receptive fields to be unevenly distributed. This distribution limits the ability of fuzzy Kanerva Coding to achieve better results. We demonstrate that another advantage of fuzzy Kanerva Coding is that it allows prototypes to tune their receptive fields for a target application. We conclude that fuzzy Kanerva Coding with prototype tuning and adaptation can significantly improve a reinforcement learner's ability to solve large-scale multi-agent problems.