The GREC main subject reference generation challenge 2009: overview and evaluation results

Authors:
Anja Belz;Eric Kow;Jette Viethen;Albert Gatt
Affiliations:
University of Brighton, Brighton, UK;University of Brighton, Brighton, UK;Macquarie University, Sydney, NSW;University of Aberdeen, Aberdeen, UK
Venue:
UCNLG+Sum '09 Proceedings of the 2009 Workshop on Language Generation and Summarisation
Year:
2009

Citing 5
Cited 2

A model-theoretic coreference scoring scheme

MUC6 '95 Proceedings of the 6th conference on Message understanding
Using semantic relations to improve information retrieval

Using semantic relations to improve information retrieval
On coreference resolution performance metrics

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Two uses of anaphora resolution in summarization

Information Processing and Management: an International Journal
Generation of repeated references to discourse entities

ENLG '07 Proceedings of the Eleventh European Workshop on Natural Language Generation

Aligning predicate argument structures in monolingual comparable texts: a new corpus for a new task

SemEval '12 Proceedings of the First Joint Conference on Lexical and Computational Semantics - Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation
Aligning predicates across monolingual comparable texts using graph-based clustering

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

The GREC-MSR Task at Generation Challenges 2009 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted one system each, and we additionally created four baseline systems. Systems were tested automatically using existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition, systems were tested in an intrinsic evaluation involving human judges. This report describes the GREC-MSR Task and the evaluation methods applied, gives brief descriptions of the participating systems, and presents the evaluation results.