ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Predicting the outcome of patients with subarachnoid hemorrhage using machine learning techniques
IEEE Transactions on Information Technology in Biomedicine - Special section on computational intelligence in medical systems
Supporting Collaborative Learning and E-Discussions Using Artificial Intelligence Techniques
International Journal of Artificial Intelligence in Education
| Hi-index | 0.00 |
Hit ratio is currently the most common metric for measuring the accuracy of classifiers. However, it doesn't compensate for classifications that might have been due to chance. The problem's magnitude is studied here through an empirical experiment on three multivalued UCI (University of California, Irvine) classification data sets, using two well-known machine learning models: C4.5 and naive Bayes. The author shows that using hit ratio can lead to erroneous conclusions. He proposes using Cohen's kappa, as a statistically robust alternative that takes random hits into account.Like any other metric, Cohen's kappa has its own shortcomings, but the author proposes that unless a better simple alternative is found, it should be mandatory in any scientific report about classifier accuracy.