Finite-time Analysis of the Multiarmed Bandit Problem
Machine Learning
Combining online and offline knowledge in UCT
Proceedings of the 24th international conference on Machine learning
Bandit-based optimization on graphs with application to library performance tuning
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Optimal robust expensive optimization is tractable
Proceedings of the 11th Annual conference on Genetic and evolutionary computation
Efficient selectivity and backup operators in Monte-Carlo tree search
CG'06 Proceedings of the 5th international conference on Computers and games
Bandit based monte-carlo planning
ECML'06 Proceedings of the 17th European conference on Machine Learning
Creating an upper-confidence-tree program for havannah
ACG'09 Proceedings of the 12th international conference on Advances in Computer Games
Scalability and parallelization of Monte-Carlo tree search
CG'10 Proceedings of the 7th international conference on Computers and games
Principled method for exploiting opening books
CG'10 Proceedings of the 7th international conference on Computers and games
| Hi-index | 0.00 |
Monte-Carlo Tree Search algorithms (MCTS [4,6]), including upper confidence trees (UCT [9]), are known for their impressive ability in high dimensional control problems. Whilst the main testbed is the game of Go, there are increasingly many applications [13,12,7]; these algorithms are now widely accepted as strong candidates for highdimensional control applications. Unfortunately, it is known that for optimal performance on a given problem, MCTS requires some tuning; this tuning is often handcrafted or automated, with in some cases a loss of consistency, i.e. a bad behavior asymptotically in the computational power. This highly undesirable property led to a stupid behavior of our main MCTS program MoGo in a real-world situation described in section This is a big trouble for our several works on automatic parameter tuning [3] and the genetic programming of new features in MoGo. We will see in this paper: - A theoretical analysis of MCTS consistency; - Detailed examples of consistent and inconsistent known algorithms; - How to modify a MCTS implementation in order to ensure consistency, independently of themodifications to the "scoring"module (the module which is automatically tuned and genetically programmed in MoGo); - As a by product of this work, we'll see the interesting property that some heavily tuned MCTS implementations are better than UCT in the sense that they do not visit the complete tree (whereas UCT asymptotically does), whilst preserving the consistency at least if "consistency" modifications above have been made.