Solving very large weakly coupled Markov decision processes
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
How to dynamically merge Markov decision processes
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Internet traffic: periodicity, tail behavior, and performance implications
System performance evaluation
The Vision of Autonomic Computing
Computer
Efficient Reinforcement Learning in Factored MDPs
IJCAI '99 Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence
Coordinated Reinforcement Learning
ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Utility Functions in Autonomic Systems
ICAC '04 Proceedings of the First International Conference on Autonomic Computing
Hierarchical reinforcement learning with the MAXQ value function decomposition
Journal of Artificial Intelligence Research
Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies
IEEE Internet Computing
Workstation capacity tuning using reinforcement learning
Proceedings of the 2007 ACM/IEEE conference on Supercomputing
Self-Optimizing Memory Controllers: A Reinforcement Learning Approach
ISCA '08 Proceedings of the 35th Annual International Symposium on Computer Architecture
Journal of Systems and Software
Boosting the performance of computing systems through adaptive configuration tuning
Proceedings of the 2009 ACM symposium on Applied Computing
VCONF: a reinforcement learning approach to virtual machines auto-configuration
ICAC '09 Proceedings of the 6th international conference on Autonomic computing
A multi-agent learning approach to online distributed resource allocation
IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
A novel multi-agent reinforcement learning approach for job scheduling in Grid computing
Future Generation Computer Systems
Smart data structures: an online machine learning approach to multicore data structures
Proceedings of the 8th ACM international conference on Autonomic computing
Improvement of systems management policies using hybrid reinforcement learning
ECML'06 Proceedings of the 17th European conference on Machine Learning
URL: A unified reinforcement learning approach for autonomic cloud management
Journal of Parallel and Distributed Computing
Online Localized Resource Allocation Application to Urban Parking Management
WI-IAT '12 Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 02
Group-based memory oversubscription for virtualized clouds
Journal of Parallel and Distributed Computing
Hi-index | 0.00 |
This paper considers a novel application domain for reinforcement learning: that of "autonomic computing," i.e. selfmanaging computing systems. RL is applied to an online resource allocation task in a distributed multi-application computing environment with independent time-varying load in each application. The task is to allocate servers in real time so as to maximize the sum of performance-based expected utility in each application. This task may be treated as a composite MDP, and to exploit the problem structure, a simple localized RL approach is proposed, with better scalability than previous approaches. The RL approach is tested in a realistic prototype data center comprising real servers, real HTTP requests, and realistic time-varying demand. This domain poses a number of major challenges associated with live training in a real system, including: the need for rapid training, exploration that avoids excessive penalties, and handling complex, potentially non-Markovian system effects. The early results are encouraging: in overnight training, RL performs as well as or slightly better than heavily researched model-based approaches derived from queuing theory.