CoNLL-2012 shared task: Modeling Multilingual Unrestricted Coreference in OntoNotes

  • Authors:
  • Sameer Pradhan;Alessandro Moschitti;Nianwen Xue;Olga Uryupina;Yuchen Zhang

  • Affiliations:
  • Raytheon BBN Technologies, Cambridge, MA;University of Trento, Povo (TN) Italy;Brandeis University, Waltham, MA;University of Trento, Povo (TN) Italy;Brandeis University, Waltham, MA

  • Venue:
  • CoNLL '12 Joint Conference on EMNLP and CoNLL - Shared Task
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

The CoNLL-2012 shared task involved predicting coreference in three languages -- English, Chinese and Arabic -- using OntoNotes data. It was a follow-on to the English-only task organized in 2011. Until the creation of the OntoNotes corpus, resources in this subfield of language processing have tended to be limited to noun phrase coreference, often on a restricted set of entities, such as ACE entities. OntoNotes provides a large-scale corpus of general anaphoric coreference not restricted to noun phrases or to a specified set of entity types and covering multiple languages. OntoNotes also provides additional layers of integrated annotation, capturing additional shallow semantic structure. This paper briefly describes the OntoNotes annotation (coreference and other layers) and then describes the parameters of the shared task including the format, pre-processing information, evaluation criteria, and presents and discusses the results achieved by the participating systems. Being a task that has a complex evaluation history, and multiple evalation conditions, it has, in the past, been difficult to judge the improvement in new algorithms over previously reported results. Having a standard test set and evaluation parameters, all based on a resource that provides multiple integrated annotation layers (parses, semantic roles, word senses, named entities and coreference) that could support joint models, should help to energize ongoing research in the task of entity and event coreference.