Comparison of classification accuracy using Cohen's Weighted Kappa

Authors:
Arie Ben-David
Affiliations:
Management Information Systems, Department of Technology Management, Holon Institute of Technology, Holon, Israel
Venue:
Expert Systems with Applications: An International Journal
Year:
2008

Citing 8
Cited 8

Cost-sensitive pruning of decision trees

ECML-94 Proceedings of the European conference on machine learning on Machine Learning
Monotonicity Maintenance in Information-Theoretic Machine Learning Algorithms

Machine Learning
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
Combined 5 × 2 cv F test for comparing supervised classification learning algorithms

Neural Computation
A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms

Machine Learning
Pruning Decision Trees with Misclassification Costs

ECML '98 Proceedings of the 10th European Conference on Machine Learning
Bootstrap Methods for the Cost-Sensitive Evaluation of Classifiers

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
On the Definition and Representation of a Ranking

ReIMICS '01 Revised Papers from the 6th International Conference and 1st Workshop of COST Action 274 TARSKI on Relational Methods in Computer Science

Sparse episode identification in environmental datasets: The case of air quality assessment

Expert Systems with Applications: An International Journal
Memetic evolutionary multi-objective neural network classifier to predict graft survival in liver transplant patients

Proceedings of the 13th annual conference companion on Genetic and evolutionary computation
Classifier ensemble construction with rotation forest to improve medical diagnosis performance of machine learning algorithms

Computer Methods and Programs in Biomedicine
A Robust Multi-Class Feature Selection Strategy Based on Rotation Forest Ensemble Algorithm for Diagnosis of Erythemato-Squamous Diseases

Journal of Medical Systems
genEnsemble: A new model for the combination of classifiers and integration of biological knowledge applied to genomic data

Expert Systems with Applications: An International Journal
The problem with kappa

EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
A random forest classifier for lymph diseases

Computer Methods and Programs in Biomedicine
A novel ensemble of classifiers that use biological relevant gene sets for microarray classification

Applied Soft Computing

Quantified Score

Hi-index	12.05

Visualization

Abstract

Many expert systems solve classification problems. While comparing the accuracy of such classifiers, the cost of error must frequently be taken into account. In such cost-sensitive applications just using the percentage of misses as the sole meter for accuracy can be misleading. Typical examples of such problems are medical and military applications, as well as data sets with ordinal (i.e., ordered) class. A new methodology is proposed here for assessing classifiers accuracy. The approach taken is based on Cohen's Kappa statistic. It compensates for classifications that may be due to chance. The use of Kappa is proposed as a standard meter for measuring the accuracy of all multi-valued classification problems. The use of Weighted Kappa enables to effectively deal with cost-sensitive classification. When the cost of error is unknown and can only be roughly estimated, the use of sensitivity analysis with Weighted Kappa is highly recommended.