A methodology for comparing classification methods through the assessment of model stability and validity in variable selection

Authors:
J. Shreve;H. Schneider;O. Soysal
Affiliations:
-;-;-
Venue:
Decision Support Systems
Year:
2011

Citing 16
Cited 1

Bankruptcy prediction using neural networks

Decision Support Systems - Special issue on neural networks for decision support
Recommender systems

Communications of the ACM
A comparison of discriminant procedures for binary variables

Computational Statistics & Data Analysis
Information Retrieval

Information Retrieval
Linkage and Autocorrelation Cause Feature Selection Bias in Relational Learning

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
A comparative assessment of classification methods

Decision Support Systems
An intelligent system for customer targeting: a data mining approach

Decision Support Systems
A principled approach for building and evaluating neural network classification models

Decision Support Systems
Toward Integrating Feature Selection Algorithms for Classification and Clustering

IEEE Transactions on Knowledge and Data Engineering
Feature subset selection bias for classification learning

ICML '06 Proceedings of the 23rd international conference on Machine learning
Competing on Analytics: The New Science of Winning

Competing on Analytics: The New Science of Winning
Learning from Imbalanced Data

IEEE Transactions on Knowledge and Data Engineering
Predicting and explaining patronage behavior toward web and traditional stores using neural networks: a comparative analysis with logistic regression

Decision Support Systems
Toward a successful CRM: variable selection, sampling, and ensemble

Decision Support Systems
Data mining for credit card fraud: A comparative study

Decision Support Systems
Considerations of sample and feature size

IEEE Transactions on Information Theory

Evaluation and aggregation of pay-as-you-drive insurance rate factors: A classification analysis approach

Decision Support Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Classification analysis utilizes features for separating observations into distinct groups for decision-making purposes. This study provides a systematic design for comparing the performance of six classification methods using Monte Carlo simulations and illustrates that the variable selection process is integral in comparing methodologies to ensure minimal bias, enhanced stability, and optimize performance. We quantify the variable selection bias and show that, for sufficiently large samples, this bias is minimized so that methods can be compared. We address topics relevant to model building and provide prescriptions for future comparisons so as to build a body of evidence for recommending their use.