Meta-evaluation of summaries in a cross-lingual environment using content-based metrics

Authors:
Horacio Saggion;Simone Teufel;Dragomir Radev;Wai Lam
Affiliations:
University of Sheffield, Sheffield, England, UK;Cambridge University, Cambridge, UK;University of Michigan, Ann Arbor, MI;The Chinese University of Hong Kong, Shatin, Hong Kong
Venue:
COLING '02 Proceedings of the 19th international conference on Computational linguistics - Volume 1
Year:
2002

Citing 10
Cited 6

Automatic text processing

Automatic text processing
Text algorithms

Text algorithms
Summarizing Similarities and Differences Among Related Documents

Information Retrieval
Evaluating Natural Language Processing Systems: An Analysis and Review

Evaluating Natural Language Processing Systems: An Analysis and Review
Cut and paste based text summarization

NAACL 2000 Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference
Tagging sentence boundaries

NAACL 2000 Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference
A program for aligning sentences in bilingual corpora

ACL '91 Proceedings of the 29th annual meeting on Association for Computational Linguistics
Concept identification and presentation in the context of technical text summarization

NAACL-ANLP-AutoSum '00 Proceedings of the 2000 NAACL-ANLPWorkshop on Automatic summarization - Volume 4
Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies

NAACL-ANLP-AutoSum '00 Proceedings of the 2000 NAACL-ANLPWorkshop on Automatic summarization - Volume 4
A comparison of rankings produced by summarization evaluation measures

NAACL-ANLP-AutoSum '00 Proceedings of the 2000 NAACL-ANLPWorkshop on Automatic summarization - Volume 4

Robust generic and query-based summarisation

EACL '03 Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - Volume 2
Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics

ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
Extractive spoken document summarization for information retrieval

Pattern Recognition Letters
Evaluating N-gram based evaluation metrics for automatic keyphrase extraction

COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics
Multilingual summarization evaluation without human models

COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics: Posters
Experiments on statistical and pattern-based biographical summarization

EPIA'05 Proceedings of the 12th Portuguese conference on Progress in Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe a framework for the evaluation of summaries in English and Chinese using similarity measures. The framework can be used to evaluate extractive, non-extractive, single and multi-document summarization. We focus on the resources developed that are made available for the research community.