Evaluation metrics for end-to-end coreference resolution systems

Authors:
Jie Cai;Michael Strube
Affiliations:
Heidelberg Institute for Theoretical Studies gGmbH, Heidelberg, Germany;Heidelberg Institute for Theoretical Studies gGmbH, Heidelberg, Germany
Venue:
SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Year:
2010

Citing 12
Cited 9

A machine learning approach to coreference resolution of noun phrases

Computational Linguistics - Special issue on computational anaphora resolution
A model-theoretic coreference scoring scheme

MUC6 '95 Proceedings of the 6th conference on Message understanding
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
A mention-synchronous coreference resolution algorithm based on the Bell tree

ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
On coreference resolution performance metrics

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution

HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
A twin-candidate model for learning-based anaphora resolution

Computational Linguistics
BART: a modular toolkit for coreference resolution

HLT-Demonstrations '08 Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Demo Session
BestCut: a graph algorithm for coreference resolution

EMNLP '06 Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing
Understanding the value of features for coreference resolution

EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Conundrums in noun phrase coreference resolution: making sense of the state-of-the-art

ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2
Supervised models for coreference resolution

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2 - Volume 2

End-to-end coreference resolution via hypergraph partitioning

COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics
Narrowing the modeling gap: a cluster-ranking approach to coreference resolution

Journal of Artificial Intelligence Research
CoNLL-2011 shared task: modeling unrestricted coreference in OntoNotes

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
RelaxCor participation in CoNLL shared task on coreference resolution

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
An incremental model for coreference resolution with restrictive antecedent accessibility

CONLL Shared Task '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Blanc: Implementing the rand index for coreference evaluation

Natural Language Engineering
Lexical patterns, features and knowledge resources for coreference resolution in clinical notes

Journal of Biomedical Informatics
Entity clustering across languages

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Coreference resolution: an empirical study based on SemEval-2010 shared Task 1

Language Resources and Evaluation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Commonly used coreference resolution evaluation metrics can only be applied to key mentions, i.e. already annotated mentions. We here propose two variants of the B3 and CEAF coreference resolution evaluation algorithms which can be applied to coreference resolution systems dealing with system mentions, i.e. automatically determined mentions. Our experiments show that our variants lead to intuitive and reliable results.