Bayesian optimization in high dimensions via random embeddings

Authors:
Ziyu Wang;Masrour Zoghi;Frank Hutter;David Matheson;Nando De Freitas
Affiliations:
University of British Columbia, Canada;University of Amsterdam, the Netherlands;Freiburg University, Germany;University of British Columbia, Canada;University of British Columbia, Canada
Venue:
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Year:
2013

Citing 14
Cited 0

Lipschitzian optimization without the Lipschitz constant

Journal of Optimization Theory and Applications
Efficient Global Optimization of Expensive Black-Box Functions

Journal of Global Optimization
A Taxonomy of Global Optimization Methods Based on Response Surfaces

Journal of Global Optimization
Completely Derandomized Self-Adaptation in Evolution Strategies

Evolutionary Computation
Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)

Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)
Practical bayesian optimization

Practical bayesian optimization
Connections in networks: a hybrid approach

CPAIOR'08 Proceedings of the 5th international conference on Integration of AI and OR techniques in constraint programming for combinatorial optimization problems
Programming by optimization

Communications of the ACM
Convergence Rates of Efficient Global Optimization Algorithms

The Journal of Machine Learning Research
Automated configuration of mixed integer programming solvers

CPAIOR'10 Proceedings of the 7th international conference on Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems
Sequential model-based optimization for general algorithm configuration

LION'05 Proceedings of the 5th international conference on Learning and Intelligent Optimization
Random search for hyper-parameter optimization

The Journal of Machine Learning Research
An experimental methodology for response surface optimization methods

Journal of Global Optimization
Parallel algorithm configuration

LION'12 Proceedings of the 6th international conference on Learning and Intelligent Optimization

Quantified Score

Hi-index	0.00

Visualization

Abstract

Bayesian optimization techniques have been successfully applied to robotics, planning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach is restricted to problems of moderate dimension, and several workshops on Bayesian optimization have identified its scaling to high dimensions as one of the holy grails of the field. In this paper, we introduce a novel random embedding idea to attack this problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple and applies to domains with both categorical and continuous variables. The experiments demonstrate that REMBO can effectively solve high-dimensional problems, including automatic parameter configuration of a popular mixed integer linear programming solver.