The potential and limitations of automatic sentence extraction for summarization

Authors:
Chin-Yew Lin;Eduard Hovy
Affiliations:
University of Southern California/Information Sciences Institute, Marina del Rey, CA;University of Southern California/Information Sciences Institute, Marina del Rey, CA
Venue:
HLT-NAACL-DUC '03 Proceedings of the HLT-NAACL 03 on Text summarization workshop - Volume 5
Year:
2003

Citing 10
Cited 17

A trainable document summarizer

SIGIR '95 Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval
Summarizing text documents: sentence selection and evaluation metrics

Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval
The automatic construction of large-scale corpora for summarization research

Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval
New Methods in Automatic Extracting

Journal of the ACM (JACM)
Generating natural language summaries from multiple on-line sources

Computational Linguistics - Special issue on natural language generation
Multidocument summarization via information extraction

HLT '01 Proceedings of the first international conference on Human language technology research
Automatic evaluation of summaries using N-gram co-occurrence statistics

NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1
A comparison of rankings produced by summarization evaluation measures

NAACL-ANLP-AutoSum '00 Proceedings of the 2000 NAACL-ANLPWorkshop on Automatic summarization - Volume 4
Manual and automatic evaluation of summaries

AS '02 Proceedings of the ACL-02 Workshop on Automatic Summarization - Volume 4
Tracking and summarizing news on a daily basis with Columbia's Newsblaster

HLT '02 Proceedings of the second international conference on Human Language Technology Research

Topic themes for multi-document summarization

Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Improving summarization performance by sentence compression: a pilot study

AsianIR '03 Proceedings of the sixth international workshop on Information retrieval with Asian languages - Volume 11
Graph-based ranking algorithms for sentence extraction, applied to text summarization

ACLdemo '04 Proceedings of the ACL 2004 on Interactive poster and demonstration sessions
QARLA: a framework for the evaluation of text summarization systems

ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Automatic summarising: The state of the art

Information Processing and Management: an International Journal
Older versions of the ROUGEeval summarization evaluation system were easier to fool

Information Processing and Management: an International Journal
On the subjectivity of human-authored summaries*

Natural Language Engineering
The Decomposition of Human-Written Book Summaries

CICLing '09 Proceedings of the 10th International Conference on Computational Linguistics and Intelligent Text Processing
An approach to text summarization

CLIAWS3 '09 Proceedings of the Third International Workshop on Cross Lingual Information Access: Addressing the Information Need of Multilingual Societies
Using N-Grams to understand the nature of summaries

HLT-NAACL-Short '04 Proceedings of HLT-NAACL 2004: Short Papers
Using topic themes for multi-document summarization

ACM Transactions on Information Systems (TOIS)
Quantifying the limits and success of extractive summarization systems across domains

HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Summarizing textual information about locations

Proceedings of the 2nd International Conference on Computing for Geospatial Research & Applications
GEMS: generative modeling for evaluation of summaries

CICLing'10 Proceedings of the 11th international conference on Computational Linguistics and Intelligent Text Processing
Automatic extractive text summarization based on fuzzy logic: a sentence oriented approach

SEMCCO'11 Proceedings of the Second international conference on Swarm, Evolutionary, and Memetic Computing - Volume Part I
Extraction of relevant figures and tables for multi-document summarization

CICLing'12 Proceedings of the 13th international conference on Computational Linguistics and Intelligent Text Processing - Volume Part II
Review: The automatic creation of concept maps from documents written using morphologically rich languages

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we present an empirical study of the potential and limitation of sentence extraction in text summarization. Our results show that the single document generic summarization task as defined in DUC 2001 needs to be carefully refocused as reflected in the low inter-human agreement at 100-word1 (0.40 score) and high upper bound at full text2 (0.88) summaries. For 100-word summaries, the performance upper bound, 0.65, achieved oracle extracts3. Such oracle extracts show the promise of sentence extraction algorithms; however, we first need to raise inter-human agreement to be able to achieve this performance level. We show that compression is a promising direction and that the compression ratio of summaries affects average human and system performance.