Random instances of a graph coloring problem are hard
STOC '88 Proceedings of the twentieth annual ACM symposium on Theory of computing
On the facial structure of the set covering polytope dimensional linear programming
Mathematical Programming: Series A and B
Floating search methods in feature selection
Pattern Recognition Letters
What makes an optimization problem hard?
Complexity
Generating hard satisfiability problems
Artificial Intelligence - Special volume on frontiers in problem solving: phase transitions and complexity
PYTHIA: a knowledge-based system to select scientific algorithms
ACM Transactions on Mathematical Software (TOMS)
Artificial Intelligence
Approximation algorithms for bin packing: a survey
Approximation algorithms for NP-hard problems
The Racing Algorithm: Model Selection for Lazy Learners
Artificial Intelligence Review - Special issue on lazy learning
Towards a characterisation of the behaviour of stochastic local search algorithms for SAT
Artificial Intelligence
On the classification of NP-complete problems in terms of their correlation coefficient
Proceedings of the 5th Twente workshop on on Graphs and combinatorial optimization
The probabilistic relationship between the assignment and asymmetric traveling salesman problems
SODA '01 Proceedings of the twelfth annual ACM-SIAM symposium on Discrete algorithms
Heuristic Solution of Open Bin Packing Problems
Journal of Heuristics
Graph Coloring with Adaptive Evolutionary Algorithms
Journal of Heuristics
On the Hardness of the Quadratic Assignment Problem with Metaheuristics
Journal of Heuristics
Evolution towards the Maximum Clique
Journal of Global Optimization
A perspective view and survey of meta-learning
Artificial Intelligence Review
Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms
Proceedings of the 6th International Conference on Genetic Algorithms
Meta-Learning by Landmarking Various Learning Algorithms
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Modeling Building-Block Interdependency
PPSN V Proceedings of the 5th International Conference on Parallel Problem Solving from Nature
Fitness Distance Correlation and Ridge Functions
PPSN V Proceedings of the 5th International Conference on Parallel Problem Solving from Nature
The Density of States - A Measure of the Difficulty of Optimisation Problems
PPSN IV Proceedings of the 4th International Conference on Parallel Problem Solving from Nature
The Phase-Transition Niche for Evolutionary Algorithms in Timetabling
Selected papers from the First International Conference on Practice and Theory of Automated Timetabling
Some Observations about GA-Based Exam Timetabling
PATAT '97 Selected papers from the Second International Conference on Practice and Theory of Automated Timetabling II
Some Complexity Aspects of Secondary School Timetabling Problems
PATAT '00 Selected papers from the Third International Conference on Practice and Theory of Automated Timetabling III
Approximation on the Web: A Compendium of NP Optimization Problems
RANDOM '97 Proceedings of the International Workshop on Randomization and Approximation Techniques in Computer Science
A Bayesian Approach to Tackling Hard Computational Problems
UAI '01 Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence
Learning the Empirical Hardness of Optimization Problems: The Case of Combinatorial Auctions
CP '02 Proceedings of the 8th International Conference on Principles and Practice of Constraint Programming
Discrete Applied Mathematics - Workshop on discrete optimization DO'99, contributions to discrete optimization
Dynamic Programming and Strong Bounds for the 0-1 Knapsack Problem
Management Science
On Graph Partitioning, Spectral Analysis, and Digital Mesh Processing
SMI '03 Proceedings of the Shape Modeling International 2003
Szemeredi''s Regularity Lemma and its applications in graph theory
Szemeredi''s Regularity Lemma and its applications in graph theory
An introduction to variable and feature selection
The Journal of Machine Learning Research
Advanced fitness landscape analysis and the performance of memetic algorithms
Evolutionary Computation - Special issue on magnetic algorithms
Phase Transitions in Combinatorial Optimization Problems - Basics, Algorithms and Statistical Mechanics
Where are the hard knapsack problems?
Computers and Operations Research
Objective Function Features Providing Barriers to Rapid Global Optimization
Journal of Global Optimization
Solving Lift-and-Project Relaxations of Binary Integer Programs
SIAM Journal on Optimization
A combinatorial characterization of the testable graph properties: it's all about regularity
Proceedings of the thirty-eighth annual ACM symposium on Theory of computing
Graph Theory and Its Applications, Second Edition (Discrete Mathematics and Its Applications)
Graph Theory and Its Applications, Second Edition (Discrete Mathematics and Its Applications)
Learning dynamic algorithm portfolios
Annals of Mathematics and Artificial Intelligence
Evolving combinatorial problem instances that are difficult to solve
Evolutionary Computation
A review of metrics on permutations for search landscape analysis
Computers and Operations Research
On the futility of blind search: An algorithmic view of “no free lunch”
Evolutionary Computation
Multi-objective genetic algorithms: Problem difficulties and construction of test problems
Evolutionary Computation
Cross-disciplinary perspectives on meta-learning for algorithm selection
ACM Computing Surveys (CSUR)
Different metaheuristic strategies to solve the feature selection problem
Pattern Recognition Letters
Problem difficulty analysis for particle swarm optimization: deception and modality
Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation
An experimental investigation of model-based parameter optimisation: SPO and beyond
Proceedings of the 11th Annual conference on Genetic and evolutionary computation
Synthetic Optimization Problem Generation: Show Us the Correlations!
INFORMS Journal on Computing
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
Combining multiple heuristics online
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Phase transitions and backbones of the asymmetric traveling salesman problem
Journal of Artificial Intelligence Research
A portfolio approach to algorithm select
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Where the really hard problems are
IJCAI'91 Proceedings of the 12th international joint conference on Artificial intelligence - Volume 1
Backbones in optimization and approximation
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 1
The backbone of the travelling salesperson
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
On learning algorithm selection for classification
Applied Soft Computing
ParamILS: an automatic algorithm configuration framework
Journal of Artificial Intelligence Research
INFORMS Journal on Computing
GECCO'03 Proceedings of the 2003 international conference on Genetic and evolutionary computation: PartII
An analysis of problem difficulty for a class of optimisation heuristics
EvoCOP'07 Proceedings of the 7th European conference on Evolutionary computation in combinatorial optimization
SATzilla-07: the design and analysis of an algorithm portfolio for SAT
CP'07 Proceedings of the 13th international conference on Principles and practice of constraint programming
Optimisation and generalisation: footprints in instance space
PPSN'10 Proceedings of the 11th international conference on Parallel problem solving from nature: Part I
Understanding TSP difficulty by learning from evolved instances
LION'10 Proceedings of the 4th international conference on Learning and intelligent optimization
Feature selection by maximum marginal diversity: optimality and implications for visual recognition
CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition
Discovering the suitability of optimisation algorithms by learning from evolved instances
Annals of Mathematics and Artificial Intelligence
Property analysis of symmetric travelling salesman problem instances acquired through evolution
EvoCOP'05 Proceedings of the 5th European conference on Evolutionary Computation in Combinatorial Optimization
Application of the grouping genetic algorithm to university course timetabling
EvoCOP'05 Proceedings of the 5th European conference on Evolutionary Computation in Combinatorial Optimization
Formulations and reformulations in integer programming
CPAIOR'05 Proceedings of the Second international conference on Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems
Generalising algorithm performance in instance space: a timetabling case study
LION'05 Proceedings of the 5th international conference on Learning and Intelligent Optimization
No free lunch theorems for optimization
IEEE Transactions on Evolutionary Computation
Fitness landscape analysis and memetic algorithms for the quadratic assignment problem
IEEE Transactions on Evolutionary Computation
Multidimensional Knapsack Problem: A Fitness Landscape Analysis
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Copositive and semidefinite relaxations of the quadratic assignment problem
Discrete Optimization
An argument for abandoning the travelling salesman problem as a neural-network benchmark
IEEE Transactions on Neural Networks
Discovering the suitability of optimisation algorithms by learning from evolved instances
Annals of Mathematics and Artificial Intelligence
A global search framework for practical three-dimensional packing with variable carton orientations
Computers and Operations Research
Proceedings of the 14th annual conference companion on Genetic and evolutionary computation
Length scale for characterising continuous optimization problems
PPSN'12 Proceedings of the 12th international conference on Parallel Problem Solving from Nature - Volume Part I
ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part III
Proceedings of the twelfth workshop on Foundations of genetic algorithms XII
Recent advances in problem understanding: changes in the landscape a year on
Proceedings of the 15th annual conference companion on Genetic and evolutionary computation
Algorithm runtime prediction: Methods & evaluation
Artificial Intelligence
Towards objective measures of algorithm performance across instance space
Computers and Operations Research
Annals of Mathematics and Artificial Intelligence
| Hi-index | 0.01 |
Discovering the conditions under which an optimization algorithm or search heuristic will succeed or fail is critical for understanding the strengths and weaknesses of different algorithms, and for automated algorithm selection. Large scale experimental studies - studying the performance of a variety of optimization algorithms across a large collection of diverse problem instances - provide the resources to derive these conditions. Data mining techniques can be used to learn the relationships between the critical features of the instances and the performance of algorithms. This paper discusses how we can adequately characterize the features of a problem instance that have impact on difficulty in terms of algorithmic performance, and how such features can be defined and measured for various optimization problems. We provide a comprehensive survey of the research field with a focus on six combinatorial optimization problems: assignment, traveling salesman, and knapsack problems, bin-packing, graph coloring, and timetabling. For these problems - which are important abstractions of many real-world problems - we review hardness-revealing features as developed over decades of research, and we discuss the suitability of more problem-independent landscape metrics. We discuss how the features developed for one problem may be transferred to study related problems exhibiting similar structures.