Hierarchical Clustering Algorithms for Document Datasets
Data Mining and Knowledge Discovery
NAACL-Short '06 Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers
Semeval-2007 task 02: evaluating word sense induction and discrimination systems
SemEval '07 Proceedings of the 4th International Workshop on Semantic Evaluations
SemEval-2010 task 14: Word sense induction & disambiguation
SemEval '10 Proceedings of the 5th International Workshop on Semantic Evaluation
KSU KDD: Word sense induction by clustering in topic space
SemEval '10 Proceedings of the 5th International Workshop on Semantic Evaluation
A quantitative evaluation of global word sense induction
CICLing'11 Proceedings of the 12th international conference on Computational linguistics and intelligent text processing - Volume Part I
Measuring the impact of sense similarity on word sense induction
EMNLP '11 Proceedings of the First Workshop on Unsupervised Learning in NLP
Evaluation of clustering algorithms for word sense disambiguation
International Journal of Data Analysis Techniques and Strategies
| Hi-index | 0.00 |
This paper presents the evaluation setting for the SemEval-2010 Word Sense Induction (WSI) task. The setting of the SemEval-2007 WSI task consists of two evaluation schemes, i.e. unsupervised evaluation and supervised evaluation. The first one evaluates WSI methods in a similar fashion to Information Retrieval exercises using F-Score. However, F-Score suffers from the matching problem which does not allow: (1) the assessment of the entire membership of clusters, and (2) the evaluation of all clusters in a given solution. In this paper, we present the use of V-measure as a measure of objectively assessing WSI methods in an unsupervised setting, and we also suggest a small modification on the supervised evaluation.