Introduction to the special issue on summarization
Computational Linguistics - Summarization
Generic technologies for single- and multi-document summarization
Information Processing and Management: an International Journal - Special issue: Cross-language information retrieval
Automatic evaluation of summaries using N-gram co-occurrence statistics
NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1
Improving summarization performance by sentence compression: a pilot study
AsianIR '03 Proceedings of the sixth international workshop on Information retrieval with Asian languages - Volume 11
Examining the consensus between human summaries: initial experiments with factoid analysis
HLT-NAACL-DUC '03 Proceedings of the HLT-NAACL 03 on Text summarization workshop - Volume 5
The potential and limitations of automatic sentence extraction for summarization
HLT-NAACL-DUC '03 Proceedings of the HLT-NAACL 03 on Text summarization workshop - Volume 5
Probabilistic model for definitional question answering
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Automatic summarising: The state of the art
Information Processing and Management: an International Journal
Older versions of the ROUGEeval summarization evaluation system were easier to fool
Information Processing and Management: an International Journal
Developing learning strategies for topic-based summarization
Proceedings of the sixteenth ACM conference on Conference on information and knowledge management
Summarization system evaluation revisited: N-gram graphs
ACM Transactions on Speech and Language Processing (TSLP)
CorrefSum: Referencial Cohesion Recovery in Extractive Summaries
PROPOR '08 Proceedings of the 8th international conference on Computational Processing of the Portuguese Language
Mind the gap: dangers of divorcing evaluations of summary content from linguistic quality
COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
Summarization with a joint model for sentence extraction and compression
ILP '09 Proceedings of the Workshop on Integer Linear Programming for Natural Langauge Processing
Multi-document summarization by maximizing informative content-words
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Journal of Biomedical Informatics
Evalinitiatives '03 Proceedings of the EACL 2003 Workshop on Evaluation Initiatives in Natural Language Processing: are evaluation methods, metrics and resources reusable?
Complex question answering: unsupervised learning approaches and experiments
Journal of Artificial Intelligence Research
Focused multi-document summarization: human summarization activity vs. automated systems techniques
Journal of Computing Sciences in Colleges
Fuzzy swarm diversity hybrid model for text summarization
Information Processing and Management: an International Journal
Formal and functional assessment of the pyramid method for summary content evaluation*
Natural Language Engineering
Discourse indicators for content selection in summarization
SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Applying regression models to query-focused multi-document summarization
Information Processing and Management: an International Journal
Heuristics based automatic text summarization of unstructured text
Proceedings of the International Conference & Workshop on Emerging Trends in Technology
Learning from collective human behavior to introduce diversity in lexical choice
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Towards a unified approach for opinion question answering and summarization
WASSA '11 Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis
Text specificity and impact on quality of news summaries
MTTG '11 Proceedings of the Workshop on Monolingual Text-To-Text Generation
Integer linear programming for dutch sentence compression
CICLing'10 Proceedings of the 11th international conference on Computational Linguistics and Intelligent Text Processing
GEMS: generative modeling for evaluation of summaries
CICLing'10 Proceedings of the 11th international conference on Computational Linguistics and Intelligent Text Processing
EM clustering algorithm for automatic text summarization
MICAI'11 Proceedings of the 10th Mexican international conference on Advances in Artificial Intelligence - Volume Part I
Degree centrality for semantic abstraction summarization of therapeutic studies
Journal of Biomedical Informatics
Summarisation of the logical structure of XML documents
Information Processing and Management: an International Journal
Entity-centric topic-oriented opinion summarization in twitter
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Multi-document summarization via submodularity
Applied Intelligence
Describing video contents in natural language
HYBRID '12 Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data
A knowledge induced graph-theoretical model for extract and abstract single document summarization
CICLing'13 Proceedings of the 14th international conference on Computational Linguistics and Intelligent Text Processing - Volume 2
Summary evaluation: together we stand NPowER-ed
CICLing'13 Proceedings of the 14th international conference on Computational Linguistics and Intelligent Text Processing - Volume 2
Semisupervised learning based opinion summarization and classification for online product reviews
Applied Computational Intelligence and Soft Computing
| Hi-index | 0.00 |
In this paper we discuss manual and automatic evaluations of summaries using data from the Document Understanding Conference 2001 (DUC-2001). We first show the instability of the manual evaluation. Specifically, the low inter-human agreement indicates that more reference summaries are needed. To investigate the feasibility of automated summary evaluation based on the recent BLEU method from machine translation, we use accumulative n-gram overlap scores between system and human summaries. The initial results provide encouraging correlations with human judgments, based on the Spearman rank-order correlation coefficient. However, relative ranking of systems needs to take into account the instability.