Reactive provisioning of backend databases in shared dynamic content server clusters
ACM Transactions on Autonomous and Adaptive Systems (TAAS)
Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies
IEEE Internet Computing
Achieving Self-Management via Utility Functions
IEEE Internet Computing
Observer: keeping system models from becoming obsolete
HotAC II Hot Topics in Autonomic Computing on Hot Topics in Autonomic Computing
IEEE Transactions on Parallel and Distributed Systems
Autonomic QoS-Aware resource management in grid computing using online performance models
Proceedings of the 2nd international conference on Performance evaluation methodologies and tools
A survey of autonomic computing—degrees, models, and applications
ACM Computing Surveys (CSUR)
Journal of Systems and Software
Autonomic multi-agent management of power and performance in data centers
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems: industrial track
An adaptive middleware for supporting time-critical event response
Cluster Computing
Autonomic QoS control in enterprise Grid environments using online simulation
Journal of Systems and Software
Self-management for neural dynamics in brain-like information processing
ICAC '09 Proceedings of the 6th international conference on Autonomic computing
Automatic exploration of datacenter performance regimes
ACDC '09 Proceedings of the 1st workshop on Automated control for datacenters and clouds
A cost-sensitive adaptation engine for server consolidation of multitier applications
Proceedings of the 10th ACM/IFIP/USENIX International Conference on Middleware
Control plane algorithms targeting challenging autonomic properties in grey systems
SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Joint admission control and resource allocation in virtualized servers
Journal of Parallel and Distributed Computing
A self-optimized job scheduler for heterogeneous server clusters
JSSPP'07 Proceedings of the 13th international conference on Job scheduling strategies for parallel processing
Autonomous return on investment analysis of additional processing resources
International Journal of Autonomic Computing
A cost-sensitive adaptation engine for server consolidation of multitier applications
Middleware'09 Proceedings of the ACM/IFIP/USENIX 10th international conference on Middleware
Autonomous Agents and Multi-Agent Systems
Statistical machine learning makes automatic control practical for internet datacenters
HotCloud'09 Proceedings of the 2009 conference on Hot topics in cloud computing
FUSION: a framework for engineering self-tuning self-adaptive software systems
Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering
A novel multi-agent reinforcement learning approach for job scheduling in Grid computing
Future Generation Computer Systems
The SCADS director: scaling a distributed storage system under stringent performance requirements
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
Decision making in autonomic computing systems: comparison of approaches and techniques
Proceedings of the 8th ACM international conference on Autonomic computing
Towards autonomic software product lines
Proceedings of the 15th International Software Product Line Conference, Volume 2
Adaptive Scheduling on Power-Aware Managed Data-Centers Using Machine Learning
GRID '11 Proceedings of the 2011 IEEE/ACM 12th International Conference on Grid Computing
Improvement of systems management policies using hybrid reinforcement learning
ECML'06 Proceedings of the 17th European conference on Machine Learning
Regression-based resource provisioning for session slowdown guarantee in multi-tier Internet servers
Journal of Parallel and Distributed Computing
Autonomic multi-policy optimization in pervasive systems: Overview and evaluation
ACM Transactions on Autonomous and Adaptive Systems (TAAS) - Special section on formal methods in pervasive computing, pervasive adaptation, and self-adaptive systems: Models and algorithms
Resource-aware adaptive scheduling for mapreduce clusters
Middleware'11 Proceedings of the 12th ACM/IFIP/USENIX international conference on Middleware
Comparison of Decision-Making Strategies for Self-Optimization in Autonomic Computing Systems
ACM Transactions on Autonomous and Adaptive Systems (TAAS) - Special Section: Extended Version of SASO 2011 Best Paper
Resource-aware adaptive scheduling for MapReduce clusters
Proceedings of the 12th International Middleware Conference
Achieving autonomous power management using reinforcement learning
ACM Transactions on Design Automation of Electronic Systems (TODAES)
Autonomic Provisioning with Self-Adaptive Neural Fuzzy Control for Percentile-Based Delay Guarantee
ACM Transactions on Autonomous and Adaptive Systems (TAAS)
Prototyping Dynamic Software Product Lines to evaluate run-time reconfigurations
Science of Computer Programming
Engineering Applications of Artificial Intelligence
Hi-index | 0.00 |
Reinforcement Learning (RL) provides a promising new approach to systems performance management that differs radically from standard queuing-theoretic approaches making use of explicit system performance models. In principle, RL can automatically learn high-quality management policies without an explicit performance model or traffic model and with little or no built-in system specific knowledge. In our original work [1], [2], [3] we showed the feasibility of using online RL to learn resource valuation estimates (in lookup table form) which can be used to make high-quality server allocation decisions in a multi-application prototype Data Center scenario. The present work shows how to combine the strengths of both RL and queuing models in a hybrid approach in which RL trains offline on data collected while a queuing model policy controls the system. By training offline we avoid suffering potentially poor performance in live online training. We also now use RL to train nonlinear function approximators (e.g. multi-layer perceptrons) instead of lookup tables; this enables scaling to substantially larger state spaces. Our results now show that in both open-loop and closed-loop traffic, hybrid RL training can achieve significant performance improvements over a variety of initial model-based policies. We also find that, as expected, RL can deal effectively with both transients and switching delays, which lie outside the scope of traditional steady-state queuing theory.