C4.5: programs for machine learning
C4.5: programs for machine learning
A non-projective dependency parser
ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Examining the consensus between human summaries: initial experiments with factoid analysis
HLT-NAACL-DUC '03 Proceedings of the HLT-NAACL 03 on Text summarization workshop - Volume 5
A metadata and annotation extractor from PDF document for semantic web
Proceedings of the 1st Amrita ACM-W Celebration on Women in Computing in India
| Hi-index | 0.00 |
This paper presents experiments with the evaluation of automatically produced summaries of literary short stories. The summaries are tailored to a particular purpose of helping a reader decide whether she wants to read the story. The evaluation procedure includes extrinsic and intrinsic measures, as well as subjective and factual judgments about the summaries pronounced by human subjects. The experiments confirm the experience of summarizing more conventional genres: sentence overlap between human- and machine-made summaries is not a complete picture of the quality of a summary. In fact, in our case, sentence overlap does not correlate well with human judgment. We explain the evaluation procedures and discuss several challenges of evaluating summaries of works of fiction.