The Vision of Autonomic Computing
Computer
ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
Coordinated Reinforcement Learning
ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
A Multi-Agent Systems Approach to Autonomic Computing
AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 1
Reinforcement Learning for Autonomic Network Repair
ICAC '04 Proceedings of the First International Conference on Autonomic Computing
Utility-Function-Driven Resource Allocation in Autonomic Systems
ICAC '05 Proceedings of the Second International Conference on Automatic Computing
A Distributed Approach for Coordination of Traffic Signal Agents
Autonomous Agents and Multi-Agent Systems
Requirements for an ubiquitous computing simulation and emulation environment
InterSense '06 Proceedings of the first international conference on Integrated internet ad hoc and sensor networks
Dealing with non-stationary environments using context detection
ICML '06 Proceedings of the 23rd international conference on Machine learning
Building autonomic systems using collaborative reinforcement learning
The Knowledge Engineering Review
Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies
IEEE Internet Computing
Batch reinforcement learning in a complex domain
Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Grid Differentiated Services: A Reinforcement Learning Approach
CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Organic Control of Traffic Lights
ATC '08 Proceedings of the 5th international conference on Autonomic and Trusted Computing
A Hybrid Reinforcement Learning Approach to Autonomic Resource Allocation
ICAC '06 Proceedings of the 2006 IEEE International Conference on Autonomic Computing
A Collaborative Reinforcement Learning Approach to Urban Traffic Control Optimization
WI-IAT '08 Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 02
Learning of coordination: exploiting sparse interactions in multiagent systems
Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
ATC '09 Proceedings of the 6th International Conference on Autonomic and Trusted Computing
Distributed W-Learning: Multi-Policy Optimization in Self-Organizing Systems
SASO '09 Proceedings of the 2009 Third IEEE International Conference on Self-Adaptive and Self-Organizing Systems
Urban traffic control structure based on hybrid Petri nets
IEEE Transactions on Intelligent Transportation Systems
| Hi-index | 0.00 |
This article describes Distributed W-Learning (DWL), a reinforcement learning-based algorithm for collaborative agent-based optimization of pervasive systems. DWL supports optimization towards multiple heterogeneous policies and addresses the challenges arising from the heterogeneity of the agents that are charged with implementing them. DWL learns and exploits the dependencies between agents and between policies to improve overall system performance. Instead of always executing the locally-best action, agents learn how their actions affect their immediate neighbors and execute actions suggested by neighboring agents if their importance exceeds the local action's importance when scaled using a predefined or learned collaboration coefficient. We have evaluated DWL in a simulation of an Urban Traffic Control (UTC) system, a canonical example of the large-scale pervasive systems that we are addressing. We show that DWL outperforms widely deployed fixed-time and simple adaptive UTC controllers under a variety of traffic loads and patterns. Our results also confirm that enabling collaboration between agents is beneficial as is the ability for agents to learn the degree to which it is appropriate for them to collaborate. These results suggest that DWL is a suitable basis for optimization in other large-scale systems with similar characteristics.