Performance Measures in Classification of Human Communications

Authors:
Marina Sokolova;Guy Lapalme
Affiliations:
Département d'informatique et de recherche opérationnelle, Université de Montréal,;Département d'informatique et de recherche opérationnelle, Université de Montréal,
Venue:
CAI '07 Proceedings of the 20th conference of the Canadian Society for Computational Studies of Intelligence on Advances in Artificial Intelligence
Year:
2007

Citing 11
Cited 1

Machine learning in automated text categorization

ACM Computing Surveys (CSUR)
Selecting the right objective measure for association analysis

Information Systems - Knowledge discovery and data mining (KDD 2002)
Text categorization with many redundant features: using aggressive feature selection to make SVMs competitive with C4.5

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Authorship verification as a one-class classification problem

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Thumbs up?: sentiment classification using machine learning techniques

EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10
Interestingness measures for data mining: A survey

ACM Computing Surveys (CSUR)
Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Learning from communication data: language in electronic business negotiations

Learning from communication data: language in electronic business negotiations
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
Get out the vote: determining support or opposition from congressional floor-debate transcripts

EMNLP '06 Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing
Constructing new and better evaluation measures for machine learning

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence

Two layered Genetic Programming for mixed-attribute data classification

Applied Soft Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This study emphasizes the importance of using appropriate measures in particular text classification settings. We focus on methods that evaluate how well a classifier performs. The effect of transformations on the confusion matrix are considered for eleven well-known and recently introduced classification measures. We analyze the measure's ability to retain its value under changes in a confusion matrix. We discuss benefits from the use of the invariant and non-invariant measures with respect to characteristics of data classes.