On learning algorithm selection for classification

Authors:
Shawkat Ali;Kate A. Smith
Affiliations:
Faculty of Information Technology, Monash University, P.O. Box 63B, Vic. 3800, Australia;Faculty of Information Technology, Monash University, P.O. Box 63B, Vic. 3800, Australia
Venue:
Applied Soft Computing
Year:
2006

Citing 18
Cited 28

Symbolic and Neural Learning Algorithms: An Experimental Comparison

Machine Learning
C4.5: programs for machine learning

C4.5: programs for machine learning
Very Simple Classification Rules Perform Well on Most Commonly Used Datasets

Machine Learning
Machine learning, neural and statistical classification

Machine learning, neural and statistical classification
Methods for comparison

Machine learning, neural and statistical classification
Evaluating and optimizing autonomous text classification systems

SIGIR '95 Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval
A note on comparing classifiers

Pattern Recognition Letters
Data mining: practical machine learning tools and techniques with Java implementations

Data mining: practical machine learning tools and techniques with Java implementations
Statistical Pattern Recognition: A Review

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms

Machine Learning
Information Retrieval

Information Retrieval
Generating Accurate Rule Sets Without Global Optimization

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Statistics for Engineering and the Sciences (5th Edition)

Statistics for Engineering and the Sciences (5th Edition)
Statistics for Business and Economics (with Student CD-ROM, iPod Key Term, and InfoTrac )

Statistics for Business and Economics (with Student CD-ROM, iPod Key Term, and InfoTrac )
A study of cross-validation and bootstrap for accuracy estimation and model selection

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
Estimating continuous distributions in Bayesian classifiers

UAI'95 Proceedings of the Eleventh conference on Uncertainty in artificial intelligence
An overview of statistical learning theory

IEEE Transactions on Neural Networks

Cross-disciplinary perspectives on meta-learning for algorithm selection

ACM Computing Surveys (CSUR)
Unsupervised Classifier Selection Based on Two-Sample Test

DS '08 Proceedings of the 11th International Conference on Discovery Science
Unique classifier selection approach for bagging algorithm

ISC '07 Proceedings of the 10th IASTED International Conference on Intelligent Systems and Control
Hybrid Repayment Prediction for Debt Portfolio

ICCCI '09 Proceedings of the 1st International Conference on Computational Collective Intelligence. Semantic Web, Social Networks and Multiagent Systems
Prediction of Sequential Values for Debt Recovery

CIARP '09 Proceedings of the 14th Iberoamerican Conference on Pattern Recognition: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications
CRBT customer churn prediction: can data mining techniques work?

International Journal of Networking and Virtual Organisations
Improved C4.5 algorithm for rule based classification

AIKED'10 Proceedings of the 9th WSEAS international conference on Artificial intelligence, knowledge engineering and data bases
Predicting vertical acceleration of railway wagons using regression algorithms

IEEE Transactions on Intelligent Transportation Systems
Symbiosis, complexification and simplicity under GP

Proceedings of the 12th annual conference on Genetic and evolutionary computation
Input space reduction for rule based classification

WSEAS Transactions on Information Science and Applications
The Effects and Interactions of Data Quality and Problem Complexity on Classification

Journal of Data and Information Quality (JDIQ)
Review: Measuring instance difficulty for combinatorial optimization problems

Computers and Operations Research
Special issue on repeatable results in software engineering prediction

Empirical Software Engineering
Automatic recommendation of classification algorithms based on data set characteristics

Pattern Recognition
SRF: a framework for the study of classifier behavior under training set mislabeling noise

PAKDD'12 Proceedings of the 16th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part I
Classifying different denial-of-service attacks in cloud computing using rule-based learning

Security and Communication Networks
Supervised pre-processing approaches in multiple class variables classification for fish recruitment forecasting

Environmental Modelling & Software
FSSC: An Algorithm for Classifying Numerical Data Using Fuzzy Soft Set Theory

International Journal of Fuzzy System Applications
Flexible Algorithm Selection Framework for Large Scale Metalearning

WI-IAT '12 Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Efficient feature size reduction via predictive forward selection

Pattern Recognition
A feature subset selection algorithm automatic recommendation method

Journal of Artificial Intelligence Research
Classification of major construction materials in construction environments using ensemble classifiers

Advanced Engineering Informatics
A metric for unsupervised metalearning

Intelligent Data Analysis
Automatic selection of classification learning algorithms for data mining practitioners

Intelligent Data Analysis
A novel feature subset selection algorithm based on association rule mining

Intelligent Data Analysis
Profit optimizing customer churn prediction with Bayesian network classifiers

Intelligent Data Analysis - Business Analytics and Intelligent Optimization
Robust classification of imbalanced data using one-class and two-class SVM-based multiclassifiers

Intelligent Data Analysis - Business Analytics and Intelligent Optimization
Alternative second-order cone programming formulations for support vector classification

Information Sciences: an International Journal

Quantified Score

Hi-index	0.02

Visualization

Abstract

This paper introduces a new method for learning algorithm evaluation and selection, with empirical results based on classification. The empirical study has been conducted among 8 algorithms/classifiers with 100 different classification problems. We evaluate the algorithms' performance in terms of a variety of accuracy and complexity measures. Consistent with the No Free Lunch theorem, we do not expect to identify the single algorithm that performs best on all datasets. Rather, we aim to determine the characteristics of datasets that lend themselves to superior modelling by certain learning algorithms. Our empirical results are used to generate rules, using the rule-based learning algorithm C5.0, to describe which types of algorithms are suited to solving which types of classification problems. Most of the rules are generated with a high confidence rating.