Empirical methods for artificial intelligence
Empirical methods for artificial intelligence
Is The Perfect The Enemy Of The Good?
GECCO '02 Proceedings of the Genetic and Evolutionary Computation Conference
A review of adaptive population sizing schemes in genetic algorithms
GECCO '05 Proceedings of the 7th annual workshop on Genetic and evolutionary computation
On initial populations of a genetic algorithm for continuous optimization problems
Journal of Global Optimization
Achieving COSMOS: a metric for determining when to give up and when to reach for the stars
Proceedings of the 14th annual conference companion on Genetic and evolutionary computation
A method to derive fixed budget results from expected optimisation times
Proceedings of the 15th annual conference on Genetic and evolutionary computation
| Hi-index | 0.00 |
In evolutionary computation, experimental results are commonly analyzed using an algorithmic performance metric called best-so-far. While best-so-far can be a useful metric, its use is particularly susceptible to three pitfalls: a failure to establish a baseline for comparison, a failure to perform significance testing, and an insufficient sample size. The nature of best-so-far means that it is highly susceptible to these pitfalls. If these pitfalls are not avoided, the use of the best-so-far metric can lead to confusion at best and misleading results at worst. We detail how the use of multiple experimental runs, random search as a baseline, and significance testing can help researchers avoid these common pitfalls. Furthermore, we demonstrate how best-so-far can be an effective algorithmic performance metric if these guidelines are followed.