No-bureaucracy evaluation

Authors:
Adam Kilgarriff
Affiliations:
University of Brighton
Venue:
Evalinitiatives '03 Proceedings of the EACL 2003 Workshop on Evaluation Initiatives in Natural Language Processing: are evaluation methods, metrics and resources reusable?
Year:
2003

Citing 3
Cited 1

Assessing agreement on classification tasks: the kappa statistic

Computational Linguistics
Introduction to the special issue on evaluating word sense disambiguation systems

Natural Language Engineering
SENSEVAL-2 Japanese translation task

SENSEVAL '01 The Proceedings of the Second International Workshop on Evaluating Word Sense Disambiguation Systems

Shared-task evaluations in HLT: lessons for NLG

INLG '06 Proceedings of the Fourth International Natural Language Generation Conference

Quantified Score

Hi-index	0.00

Visualization

Abstract

Senseval is a series of evaluation exercises for Word Sense Disambiguation. The core design is in accordance with the MUC and TREC model of quantitative, developer-oriented (rather than user-oriented) evaluation. The first was in 1998, with tasks for three languages and 25 participating research teams, the second in 2001, with tasks for twelve languages, thirty-five participating research teams and over 90 participating systems. The third is currently in planning. The scale of the resources developed is indicated in Table 1 (reproduced from (Edmonds and Kil-garriff, 2002)).