The GREC named entity generation challenge 2009: overview and evaluation results

Authors:
Anja Belz;Eric Kow;Jette Viethen
Affiliations:
University of Brighton, Brighton, UK;University of Brighton, Brighton, UK;Macquarie University, Sydney, NSW
Venue:
UCNLG+Sum '09 Proceedings of the 2009 Workshop on Language Generation and Summarisation
Year:
2009

Citing 3
Cited 3

A model-theoretic coreference scoring scheme

MUC6 '95 Proceedings of the 6th conference on Message understanding
Using semantic relations to improve information retrieval

Using semantic relations to improve information retrieval
On coreference resolution performance metrics

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing

Comparing rating scales and preference judgements in language evaluation

INLG '10 Proceedings of the 6th International Natural Language Generation Conference
UDel: refining a method of named entity generation

INLG '10 Proceedings of the 6th International Natural Language Generation Conference
Information status distinctions and referring expressions: An empirical study of references to people in news summaries

Computational Linguistics

Quantified Score

Hi-index	0.00

Visualization

Abstract

The GREC-NEG Task at Generation Challenges 2009 required participating systems to select coreference chains for all people entities mentioned in short encyclopaedic texts about people collected from Wikipedia. Three teams submitted six systems in total, and we additionally created four baseline systems. Systems were tested automatically using a range of existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition, systems were tested in an intrinsic evaluation involving human judges. This report describes the GREC-NEG Task and the evaluation methods applied, gives brief descriptions of the participating systems, and presents the evaluation results.