Relevance as a metric for evaluating machine learning algorithms

Authors:
Aravind Kota Gopalakrishna;Tanir Ozcelebi;Antonio Liotta;Johan J. Lukkien
Affiliations:
Department of Mathematics and Computer Science, System Architecture and Networking (SAN), The Netherlands;Department of Mathematics and Computer Science, System Architecture and Networking (SAN), The Netherlands;Department of Mathematics and Computer Science, System Architecture and Networking (SAN), The Netherlands,Electro-Optical Communications, Department of Electrical Engineering, Eindhoven University ...;Department of Mathematics and Computer Science, System Architecture and Networking (SAN), The Netherlands
Venue:
MLDM'13 Proceedings of the 9th international conference on Machine Learning and Data Mining in Pattern Recognition
Year:
2013

Citing 6
Cited 0

Induction of ripple-down rules applied to modeling large databases

Journal of Intelligent Information Systems
The CN2 Induction Algorithm

Machine Learning
The Power of Decision Tables

ECML '95 Proceedings of the 8th European Conference on Machine Learning
Generating Accurate Rule Sets Without Global Optimization

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
The WEKA data mining software: an update

ACM SIGKDD Explorations Newsletter
Beyond accuracy, f-score and ROC: a family of discriminant measures for performance evaluation

AI'06 Proceedings of the 19th Australian joint conference on Artificial Intelligence: advances in Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

In machine learning, the choice of a learning algorithm that is suitable for the application domain is critical. The performance metric used to compare different algorithms must also reflect the concerns of users in the application domain under consideration. In this paper, we propose a novel probability-based performance metric called Relevance Score for evaluating supervised learning algorithms. We evaluate the proposed metric through empirical analysis on a dataset gathered from an intelligent lighting pilot installation. In comparison to the commonly used Classification Accuracy metric, the Relevance Score proves to be more appropriate for a certain class of applications.