A Comparative Study of Performance Measures for Information Retrieval Systems

  • Authors:
  • Xiannong Meng

  • Affiliations:
  • Bucknell University

  • Venue:
  • ITNG '06 Proceedings of the Third International Conference on Information Technology: New Generations
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Traditional performance measures of information retrieval systems include precision and recall and their variants. While these measures work well in closed-laboratory environments, they are not suitable for practical IR systems such as Web search systems. Many single-value measures were proposed to improve over the precision-recall measure, such as expected search length (ESL), average search length (ASL) and RankPower. We compare in this paper the measures of ESL, ASL, and RankPower applied to a set of real Web retrieval data. The results demonstrate that RankPower indeed is a feasible, effective, and easyto- use single-value measure for performance of practical IR systems such as Web search engines.