The Strength of Weak Learnability
Machine Learning
High-level optimization via automated statistical modeling
PPOPP '95 Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming
Analysis of two simple heuristics on a random instance of k-SAT
Journal of Algorithms
Generating hard satisfiability problems
Artificial Intelligence - Special volume on frontiers in problem solving: phase transitions and complexity
Wrappers for feature subset selection
Artificial Intelligence - Special issue on relevance
A general upper bound for the satisfiability threshold of random r-SAT formulae
Journal of Algorithms
On the complexity of unsatisfiability proofs for random k-CNF formulas
STOC '98 Proceedings of the thirtieth annual ACM symposium on Theory of computing
Complexity analysis admissible heuristic search
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Branch and bound algorithm selection by performance prediction
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Computationally Manageable Combinational Auctions
Management Science
Towards a characterisation of the behaviour of stochastic local search algorithms for SAT
Artificial Intelligence
Typical random 3-SAT formulae and the satisfiability threshold
SODA '00 Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms
Bidding and allocation in combinatorial auctions
Proceedings of the 2nd ACM conference on Electronic commerce
Optimal solutions for multi-unit combinatorial auctions: branch and bound heuristics
Proceedings of the 2nd ACM conference on Electronic commerce
Towards a universal test suite for combinatorial auction algorithms
Proceedings of the 2nd ACM conference on Electronic commerce
Artificial Intelligence - special issue on computational tradeoffs under bounded resources
Results related to threshold phenomena research in satisfiability: lower bounds
Theoretical Computer Science - Phase transitions in combinatorial problems
Lower bounds for random 3-SAT via differential equations
Theoretical Computer Science - Phase transitions in combinatorial problems
Algorithm for optimal winner determination in combinatorial auctions
Artificial Intelligence
Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems
Journal of Automated Reasoning
Algorithm Selection using Reinforcement Learning
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Taming the Computational Complexity of Combinatorial Auctions: Optimal and Approximate Approaches
IJCAI '99 Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence
An Algorithm for Optimal Winner Determination in Combinatorial Auctions
IJCAI '99 Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence
A Bayesian Approach to Tackling Hard Computational Problems
UAI '01 Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence
Solving Combinatorial Auctions Using Stochastic Local Search
Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
An Algorithm for Multi-Unit Combinatorial Auctions
Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
Generating Satisfiable Problem Instances
Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
Phase Transitions and Backbones of 3-SAT and Maximum 3-SAT
CP '01 Proceedings of the 7th International Conference on Principles and Practice of Constraint Programming
The Adaptive Constraint Engine
CP '02 Proceedings of the 8th International Conference on Principles and Practice of Constraint Programming
Learning the Empirical Hardness of Optimization Problems: The Case of Combinatorial Auctions
CP '02 Proceedings of the 8th International Conference on Principles and Practice of Constraint Programming
Restart Policies with Dependence among Runs: A Dynamic Programming Approach
CP '02 Proceedings of the 8th International Conference on Principles and Practice of Constraint Programming
Integer Programming for Combinatorial Auction Winner Determination
ICMAS '00 Proceedings of the Fourth International Conference on MultiAgent Systems (ICMAS-2000)
Portable high-performance supercomputing: high-level platform-dependent optimization
Portable high-performance supercomputing: high-level platform-dependent optimization
Combinatorial Auctions: A Survey
INFORMS Journal on Computing
Stochastic Local Search: Foundations & Applications
Stochastic Local Search: Foundations & Applications
A sharp threshold in proof complexity yields lower bounds for satisfiability search
Journal of Computer and System Sciences - STOC 2001
Heuristic average-case analysis of the backtrack resolution of random 3-satisfiability instances
Theoretical Computer Science
Combinatorial Auctions
Learning dynamic algorithm portfolios
Annals of Mathematics and Artificial Intelligence
Measuring empirical computational complexity
Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Combining multiple heuristics online
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Hiding satisfying assignments: two are better than one
Journal of Artificial Intelligence Research
Generating hard satisfiable formulas by hiding solutions deceptively
Journal of Artificial Intelligence Research
SATzilla: portfolio-based algorithm selection for SAT
Journal of Artificial Intelligence Research
Backdoors to typical case complexity
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
A portfolio approach to algorithm select
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Constraint satisfaction, databases, and logic
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Backbones in optimization and approximation
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 1
CABOB: a fast optimal algorithm for combinatorial auctions
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Hierarchical hardness models for SAT
CP'07 Proceedings of the 13th international conference on Principles and practice of constraint programming
SATzilla-07: the design and analysis of an algorithm portfolio for SAT
CP'07 Proceedings of the 13th international conference on Principles and practice of constraint programming
Problem structure in the presence of perturbations
AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
Performance prediction and automated tuning of randomized and parametric algorithms
CP'06 Proceedings of the 12th international conference on Principles and Practice of Constraint Programming
From spin glasses to hard satisfiable formulas
SAT'04 Proceedings of the 7th international conference on Theory and Applications of Satisfiability Testing
Using CBR to select solution strategies in constraint programming
ICCBR'05 Proceedings of the 6th international conference on Case-Based Reasoning Research and Development
Bid evaluation in combinatorial auctions: optimization and learning
Software—Practice & Experience
Practical performance models of algorithms in evolutionary program induction and other domains
Artificial Intelligence
Tradeoffs in the empirical evaluation of competing algorithm designs
Annals of Mathematics and Artificial Intelligence
Selecting Simulation Algorithm Portfolios by Genetic Algorithms
PADS '10 Proceedings of the 2010 IEEE Workshop on Principles of Advanced and Distributed Simulation
Algorithms and mechanisms for procuring services with uncertain durations using redundancy
Artificial Intelligence
Discovering the suitability of optimisation algorithms by learning from evolved instances
Annals of Mathematics and Artificial Intelligence
Sequential model-based optimization for general algorithm configuration
LION'05 Proceedings of the 5th international conference on Learning and Intelligent Optimization
A meta-learning prediction model of algorithm performance for continuous optimization problems
PPSN'12 Proceedings of the 12th international conference on Parallel Problem Solving from Nature - Volume Part I
Quantifying homogeneity of instance sets for algorithm configuration
LION'12 Proceedings of the 6th international conference on Learning and Intelligent Optimization
Models of performance of time series forecasters
Neurocomputing
Algorithm runtime prediction: Methods & evaluation
Artificial Intelligence
Information Sciences: an International Journal
Hi-index | 0.01 |
Is it possible to predict how long an algorithm will take to solve a previously-unseen instance of an NP-complete problem? If so, what uses can be found for models that make such predictions? This article provides answers to these questions and evaluates the answers experimentally. We propose the use of supervised machine learning to build models that predict an algorithm's runtime given a problem instance. We discuss the construction of these models and describe techniques for interpreting them to gain understanding of the characteristics that cause instances to be hard or easy. We also present two applications of our models: building algorithm portfolios that outperform their constituent algorithms, and generating test distributions that emphasize hard problems. We demonstrate the effectiveness of our techniques in a case study of the combinatorial auction winner determination problem. Our experimental results show that we can build very accurate models of an algorithm's running time, interpret our models, build an algorithm portfolio that strongly outperforms the best single algorithm, and tune a standard benchmark suite to generate much harder problem instances.