CoNLL-2011 shared task: modeling unrestricted coreference in OntoNotes

Authors:
Sameer Pradhan;Lance Ramshaw;Mitchell Marcus;Martha Palmer;Ralph Weischedel;Nianwen Xue
Affiliations:
BBN Technologies, Cambridge, MA;BBN Technologies, Cambridge, MA;University of Pennsylvania, Philadelphia;University of Colorado, Boulder, CO;BBN Technologies, Cambridge, MA;Brandeis University, Waltham, MA
Venue:
CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Year:
2011

Citing 29
Cited 37

Building a large annotated corpus of English: the penn treebank

Computational Linguistics - Special issue on using large corpora: II
A machine learning approach to coreference resolution of noun phrases

Computational Linguistics - Special issue on computational anaphora resolution
A model-theoretic coreference scoring scheme

MUC6 '95 Proceedings of the 6th conference on Message understanding
Text and knowledge mining for coreference resolution

NAACL '01 Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies
Edit detection and parsing for transcribed speech

NAACL '01 Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies
Coreference for NLP applications

ACL '00 Proceedings of the 38th Annual Meeting on Association for Computational Linguistics
Support Vector Learning for Semantic Argument Classification

Machine Learning
The Proposition Bank: An Annotated Corpus of Semantic Roles

Computational Linguistics
Coarse-to-fine n-best parsing and MaxEnt discriminative reranking

ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Bootstrapping path-based pronoun resolution

ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
On coreference resolution performance metrics

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution

HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Unrestricted Coreference: Identifying Entities and Events in OntoNotes

ICSC '07 Proceedings of the International Conference on Semantic Computing
OntoNotes: A Unified Relational Semantic Representation

ICSC '07 Proceedings of the International Conference on Semantic Computing
The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies

CoNLL '08 Proceedings of the Twelfth Conference on Computational Natural Language Learning
The CoNLL-2009 shared task: syntactic and semantic dependencies in multiple languages

CoNLL '09 Proceedings of the Thirteenth Conference on Computational Natural Language Learning: Shared Task
The reliability of anaphoric annotation, reconsidered: taking ambiguity into account

CorpusAnno '05 Proceedings of the Workshop on Frontiers in Corpus Annotations II: Pie in the Sky
Semantic role labeling for coreference resolution

EACL '06 Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Posters & Demonstrations
OntoNotes: the 90% solution

NAACL-Short '06 Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers
Shallow semantics for coreference resolution

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Issues in synchronizing the English Treebank and PropBank

LAC '06 Proceedings of the Workshop on Frontiers in Linguistically Annotated Corpora 2006
Using decision trees for conference resolution

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
State-of-the-art NLP approaches to coreference resolution: theory and practical recipes

ACLTutorials '09 Tutorial Abstracts of ACL-IJCNLP 2009
Conundrums in noun phrase coreference resolution: making sense of the state-of-the-art

ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2
Supervised models for coreference resolution

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2 - Volume 2
Coreference resolution in a modular, entity-centered model

HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Supervised noun phrase coreference research: the first fifteen years

ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
SemEval-2010 task 1: Coreference resolution in multiple languages

SemEval '10 Proceedings of the 5th International Workshop on Semantic Evaluation
Evaluation metrics for end-to-end coreference resolution systems

SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Stanford's multi-pass sieve coreference resolution system at the CoNLL-2011 shared task

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
RelaxCor participation in CoNLL shared task on coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Inference protocols for coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Exploring lexicalized features for coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Rule and tree ensembles for unrestricted coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Unrestricted coreference resolution via global hypergraph partitioning

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Multi-metric optimization for coreference: the UniTN/IITP/Essex submission to the 2011 CoNLL Shared Task

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Combining syntactic and semantic features by SVM for unrestricted coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Supervised coreference resolution with SUCRE

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
ETS: an error tolerable system for coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Narrative schema as world knowledge for coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Hybrid approach for coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Poly-co: a multilayer perceptron approach for coreference detection

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Mention detection: heuristics for the OntoNotes annotations

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Coreference resolution with loose transitivity constraints

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
UBIU: a robust system for resolving unrestricted coreference

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Reconciling OntoNotes: unrestricted coreference resolution in OntoNotes with Reconcile

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Link type based pre-cluster pair model for coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Multiobjective simulated annealing based approach for feature selection in anaphora resolution

DAARC'11 Proceedings of the 8th international conference on Anaphora Processing and Applications
Coreference resolution using tree CRFs

CICLing'12 Proceedings of the 13th international conference on Computational Linguistics and Intelligent Text Processing - Volume Part I
Active learning for coreference resolution

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Adaptive clustering for coreference resolution with deterministic rules and web-based language models

SemEval '12 Proceedings of the First Joint Conference on Lexical and Computational Semantics - Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation
Joint entity and event coreference resolution across documents

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Resolving complex cases of definite pronouns: the winograd schema challenge

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Joint learning for coreference resolution with Markov logic

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Active learning for coreference resolution

BioNLP '12 Proceedings of the 2012 Workshop on Biomedical Natural Language Processing
Open-domain anatomical entity mention detection

ACL '12 Proceedings of the Workshop on Detecting Structure in Scholarly Discourse
Using syntactic dependencies to solve coreferences

CoNLL '12 Joint Conference on EMNLP and CoNLL - Shared Task
A mixed deterministic model for coreference resolution

CoNLL '12 Joint Conference on EMNLP and CoNLL - Shared Task
Simple maximum entropy models for multilingual coreference resolution

CoNLL '12 Joint Conference on EMNLP and CoNLL - Shared Task
UBIU for multilingual coreference resolution in OntoNotes

CoNLL '12 Joint Conference on EMNLP and CoNLL - Shared Task
Illinois-Coref: the UI system in the CoNLL-2012 shared task

CoNLL '12 Joint Conference on EMNLP and CoNLL - Shared Task
Learning to model multilingual unrestricted coreference in OntoNotes

CoNLL '12 Joint Conference on EMNLP and CoNLL - Shared Task
Phrase detectives: Utilizing collective intelligence for internet-scale language resource creation

ACM Transactions on Interactive Intelligent Systems (TiiS) - Special section on internet-scale human problem solving and regular papers
Coreference resolution: an empirical study based on SemEval-2010 shared Task 1

Language Resources and Evaluation
A constraint-based hypergraph partitioning approach to coreference resolution

Computational Linguistics
Deterministic coreference resolution based on entity-centric, precision-ranked rules

Computational Linguistics

Quantified Score

Hi-index	0.00

Visualization

Abstract

The CoNLL-2011 shared task involved predicting coreference using OntoNotes data. Resources in this field have tended to be limited to noun phrase coreference, often on a restricted set of entities, such as ace entities. OntoNotes provides a large-scale corpus of general anaphoric coreference not restricted to noun phrases or to a specified set of entity types. OntoNotes also provides additional layers of integrated annotation, capturing additional shallow semantic structure. This paper briefly describes the OntoNotes annotation (coreference and other layers) and then describes the parameters of the shared task including the format, pre-processing information, and evaluation criteria, and presents and discusses the results achieved by the participating systems. Having a standard test set and evaluation parameters, all based on a new resource that provides multiple integrated annotation layers (parses, semantic roles, word senses, named entities and coreference) that could support joint models, should help to energize ongoing research in the task of entity and event coreference.