Term Impacts as Normalized Term Frequencies for BM25 Similarity Scoring

  • Authors:

  • Vo Ngoc Anh;Raymond Wan;Alistair Moffat

  • Affiliations:

  • Department of Computer Science and Software Engineering, The University of Melbourne, Victoria, Australia 3010;Bioinformatics Center, Kyoto University, Kyoto, Japan 611-0011;Department of Computer Science and Software Engineering, The University of Melbourne, Victoria, Australia 3010

  • Venue:
  • SPIRE '08 Proceedings of the 15th International Symposium on String Processing and Information Retrieval
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

The BM25 similarity computation has been shown to provide effective document retrieval. In operational terms, the formulae which form the basis for BM25 employ both term frequency and document length normalization. This paper considers an alternative form of normalization using document-centric impacts, and shows that the new normalization simplifies BM25 and reduces the number of tuning parameters. Motivation is provided by a preliminary analysis of a document collection that shows that impacts are more likely to identify documents whose lengths resemble those of the relevant judgments.Experiments on TREC data demonstrate that impact-based BM25 is as good as or better than the original term frequency-based BM25 in terms of retrieval effectiveness.