An Evaluation of Statistical Approaches to Text Categorization
Information Retrieval
Probe, count, and classify: categorizing hidden web databases
SIGMOD '01 Proceedings of the 2001 ACM SIGMOD international conference on Management of data
Hierarchical Text Classification and Evaluation
ICDM '01 Proceedings of the 2001 IEEE International Conference on Data Mining
Using AUC and Accuracy in Evaluating Learning Algorithms
IEEE Transactions on Knowledge and Data Engineering
Introduction to Data Mining, (First Edition)
Introduction to Data Mining, (First Edition)
Hierarchical classification: combining Bayes with SVM
ICML '06 Proceedings of the 23rd international conference on Machine learning
The relationship between Precision-Recall and ROC curves
ICML '06 Proceedings of the 23rd international conference on Machine learning
Hierarchical text categorization and its application to bioinformatics
Hierarchical text categorization and its application to bioinformatics
Multi-concept Document Classification Using a Perceptron-Like Algorithm
WI-IAT '08 Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01
Exploiting known taxonomies in learning overlapping concepts
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Multi-label classification and extracting predicted class hierarchies
Pattern Recognition
A survey of hierarchical classification across different application domains
Data Mining and Knowledge Discovery
LIBSVM: A library for support vector machines
ACM Transactions on Intelligent Systems and Technology (TIST)
Hierarchical multi-classification with predictive clustering trees in functional genomics
EPIA'05 Proceedings of the 12th Portuguese conference on Progress in Artificial Intelligence
Learning classifiers using hierarchically structured class taxonomies
SARA'05 Proceedings of the 6th international conference on Abstraction, Reformulation and Approximation
| Hi-index | 0.00 |
Multi-label Classification (MC) often deals with hierarchically organized class taxonomies. In contrast to Hierarchical Multilabel Classification (HMC), where the class hierarchy is assumed to be known a priori, we are interested in the opposite case where it is unknown and should be extracted from multi-label data automatically. In this case the predictive performance of a classifier can be assessed by well-known Performance Measures (PMs) used in flat MC such as precision and recall. The fact that these PMs treat all class labels as independent labels, in contrast to hierarchically structured taxonomies, is a problem. As an alternative, special hierarchical PMs can be used that utilize hierarchy knowledge and apply this knowledge to the extracted hierarchy. This type of hierarchical PM has only recently been mentioned in literature. The aim of this study is first to verify whether HMC measures do significantly improve quality assessment in this setting. In addition, we seek to find a proper measure that reflects the potential quality of extracted hierarchies in the best possible way. We empirically compare ten hierarchical and four traditional flat PMs in order to investigate relations between them. The performance measurements obtained for predictions of four multi-label classifiers ML-ARAM, ML-kNN, BoosTexter and SVM on four datasets from the text mining domain are analyzed by means of hierarchical clustering and by calculating pairwise statistical consistency and discriminancy.