An evaluation corpus for temporal summarization

  • Authors:
  • Vikash Khandelwal;Rahul Gupta;James Allan

  • Affiliations:
  • University of Massachusetts, Amherst, MA;University of Massachusetts, Amherst, MA;University of Massachusetts, Amherst, MA

  • Venue:
  • HLT '01 Proceedings of the first international conference on Human language technology research
  • Year:
  • 2001

Quantified Score

Hi-index 0.00

Visualization

Abstract

In recent years, a lot of work has been done in the field of Topic Tracking. The focus of this work has been on identifying stories belonging to the same topic. This might result in a very large number of stories being reported to the user. It might be more useful to a user if a summary of the main events in the topic rather than the entire collection of stories related to the topic were presented. Though work on such a fine-grained level has been started, there is currently no standard evaluation testbed available to measure the accuracy of such techniques. We describe a scheme for developing a testbed of user judgments which can be used to evaluate the above mentioned techniques. The corpus that we have created can also be used to evaluate single or multi-document summaries.