How not to lie with statistics: the correct way to summarize benchmark results
Communications of the ACM - The MIT Press scientific computation series
Numerical experiments with the LANCELOT package (release A) for large-scale nonlinear optimization
Mathematical Programming: Series A and B
Experimental Evaluation of Heuristic Optimization Algorithms: A Tutorial
Journal of Heuristics
An analysis of the behavior of a class of genetic adaptive systems.
An analysis of the behavior of a class of genetic adaptive systems.
Pattern Recognition and Machine Learning (Information Science and Statistics)
Pattern Recognition and Machine Learning (Information Science and Statistics)
Stochastic algorithms assessment using performance profiles
Proceedings of the 13th annual conference on Genetic and evolutionary computation
| Hi-index | 0.00 |
One of the many difficulties that arise in the empirical evaluation of new computational techniques is the analysis and reporting of experiments involving a large number of test-problems and algorithms. The performance profiles are a methodology specifically developed for this purpose which provides a simple means of visualizing and interpreting the results of large-scale benchmarking experiments. However good, performance profiles do not take into account the uncertainty present in most experimental settings. This paper presents an extension of this analytic tool called probabilistic performance profiles. The basic idea is to endow the original performance profiles with a probabilistic interpretation, which makes it possible to represent the expected performance of a stochastic algorithm in a convenient way. The benefits of the new method are demonstrated with data from a real benchmark experiment involving several problems and algorithms.