Towards multidocument summarization by reformulation: progress and prospects
AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
Generating natural language summaries from multiple on-line sources
Computational Linguistics - Special issue on natural language generation
Probabilistic text structuring: experiments with sentence ordering
ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
Computing locally coherent discourses
ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
A bottom-up approach to sentence ordering for multi-document summarization
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Improving chronological sentence ordering by precedence relation
COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Automatic Evaluation of Information Ordering: Kendall's Tau
Computational Linguistics
Automated multi-document summarization in NeATS
HLT '02 Proceedings of the second international conference on Human Language Technology Research
Inferring strategies for sentence ordering in multidocument news summarization
Journal of Artificial Intelligence Research
A machine learning approach to sentence ordering for multidocument summarization and its evaluation
IJCNLP'05 Proceedings of the Second international joint conference on Natural Language Processing
Evaluating centering for information ordering using corpora
Computational Linguistics
A bottom-up approach to sentence ordering for multi-document summarization
Information Processing and Management: an International Journal
A model for Chinese sentence ordering based on Markov model
FSKD'09 Proceedings of the 6th international conference on Fuzzy systems and knowledge discovery - Volume 7
Multimedia news exploration and retrieval by integrating keywords, relations and visual features
Multimedia Tools and Applications
Sentence ordering driven by local and global coherence for summary generation
HLT-SS '11 Proceedings of the ACL 2011 Student Session
Extending the entity-based coherence model with multiple ranks
EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
| Hi-index | 0.00 |
The issue of sentence ordering is an important one for natural language tasks such as multi-document summarization, yet there has not been a quantitative exploration of the range of acceptable sentence orderings for short texts. We present results of a sentence reordering experiment with three experimental conditions. Our findings indicate a very high degree of variability in the orderings that the eighteen subjects produce. In addition, the variability of reorderings is significantly greater when the initial ordering seen by subjects is different from the original summary. We conclude that evaluation of sentence ordering should use multiple reference orderings. Our evaluation presents several metrics that might prove useful in assessing against multiple references. We conclude with a deeper set of questions: (a) what sorts of independent assessments of quality of the different reference orderings could be made and (b) whether a large enough test set would obviate the need for such independent means of quality assessment.