Older versions of the ROUGEeval summarization evaluation system were easier to fool

  • Authors:
  • Jonas Sjöbergh

  • Affiliations:
  • KTH CSC, SE-10044 Stockholm, Sweden

  • Venue:
  • Information Processing and Management: an International Journal
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

We show some limitations of the ROUGE evaluation method for automatic summarization. We present a method for automatic summarization based on a Markov model of the source text. By a simple greedy word selection strategy, summaries with high ROUGE-scores are generated. These summaries would however not be considered good by human readers. The method can be adapted to trick different settings of the ROUGEeval package.