Machine learning, neural and statistical classification
Machine learning, neural and statistical classification
Sampling-Based Relative Landmarks: Systematically Test-Driving Algorithms Before Choosing
EPIA '01 Proceedings of the10th Portuguese Conference on Artificial Intelligence on Progress in Artificial Intelligence, Knowledge Extraction, Multi-agent Systems, Logic Programming and Constraint Solving
Predicting relative performance of classifiers from samples
ICML '05 Proceedings of the 22nd international conference on Machine learning
The WEKA data mining software: an update
ACM SIGKDD Explorations Newsletter
An iterative process for building learning curves and predicting relative performance of classifiers
EPIA'07 Proceedings of the aritficial intelligence 13th Portuguese conference on Progress in artificial intelligence
Metalearning: Applications to Data Mining
Metalearning: Applications to Data Mining
Selecting classification algorithms with active testing
MLDM'12 Proceedings of the 8th international conference on Machine Learning and Data Mining in Pattern Recognition
| Hi-index | 0.00 |
Currently many classification algorithms exist and there is no algorithm that would outperform all the others in all tasks. Therefore it is of interest to determine which classification algorithm is the best one for a given task. Although direct comparisons can be made for any given problem using a cross-validation evaluation, it is desirable to avoid this, as the computational costs are significant. We describe a method which relies on relatively fast pairwise comparisons involving two algorithms. This method exploits sampling landmarks, that is information about learning curves besides classical data characteristics. One key feature of this method is an iterative procedure for extending the series of experiments used to gather new information in the form of sampling landmarks. Metalearning plays also a vital role. The comparisons between various pairs of algorithm are repeated and the result is represented in the form of a partially ordered ranking. Evaluation is done by comparing the partial order of algorithm that has been predicted to the partial order representing the supposedly correct result. The results of our analysis show that the method has good performance and could be of help in practical applications.