Generalizing from case studies: a case study
ML92 Proceedings of the ninth international workshop on Machine learning
Characterizing the applicability of classification algorithms using meta-level learning
ECML-94 Proceedings of the European conference on machine learning on Machine Learning
Machine learning, neural and statistical classification
Machine learning, neural and statistical classification
From data mining to knowledge discovery: an overview
Advances in knowledge discovery and data mining
The process of knowledge discovery in databases
Advances in knowledge discovery and data mining
Characterization of Classification Algorithms
EPIA '95 Proceedings of the 7th Portuguese Conference on Artificial Intelligence: Progress in Artificial Intelligence
ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Experiments in Meta-level Learning with ILP
PKDD '99 Proceedings of the Third European Conference on Principles of Data Mining and Knowledge Discovery
Reducing Rankings of Classifiers by Eliminating Redundant Classifiers
EPIA '01 Proceedings of the10th Portuguese Conference on Artificial Intelligence on Progress in Artificial Intelligence, Knowledge Extraction, Multi-agent Systems, Logic Programming and Constraint Solving
Zoomed Ranking: Selection of Classification Algorithms Based on Relevant Performance Information
PKDD '00 Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery
Discovering Task Neighbourhoods Through Landmark Learning Performances
PKDD '00 Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery
A Comparative Study of Some Issues Concerning Algorithm Recommendation Using Ranking Methods
IBERAMIA 2002 Proceedings of the 8th Ibero-American Conference on AI: Advances in Artificial Intelligence
METALA: A Meta-learning Architecture
Proceedings of the International Conference, 7th Fuzzy Days on Computational Intelligence, Theory and Applications
A New Classifier Design with Fuzzy Functions
RSFDGrC '07 Proceedings of the 11th International Conference on Rough Sets, Fuzzy Sets, Data Mining and Granular Computing
An empirical study of classification algorithm evaluation for financial risk prediction
Applied Soft Computing
Hybrid assistance in KDD task definition
SMO'05 Proceedings of the 5th WSEAS international conference on Simulation, modelling and optimization
An efficient multi-label support vector machine with a zero label
Expert Systems with Applications: An International Journal
Multi-label weighted k-nearest neighbor classifier with adaptive weight estimation
ICONIP'11 Proceedings of the 18th international conference on Neural Information Processing - Volume Part II
Pattern Recognition
Random block coordinate descent method for multi-label support vector machine with a zero label
Expert Systems with Applications: An International Journal
| Hi-index | 0.00 |
We investigate the problem of using past performance information to select an algorithm for a given classification problem. We present three ranking methods for that purpose: average ranks, success rate ratios and significant wins. We also analyze the problem of evaluating and comparing these methods. The evaluation technique used is based on a leave-one-out procedure. On each iteration, the method generates a ranking using the results obtained by the algorithms on the training datasets. This ranking is then evaluated by calculating its distance from the ideal ranking built using the performance information on the test dataset. The distance measure adopted here, average correlation, is based on Spearman's rank correlation coefficient. To compare ranking methods, a combination of Friedman's test and Dunn's multiple comparison procedure is adopted. When applied to the methods presented here, these tests indicate that the success rate ratios and average ranks methods perform better than significant wins.