On rank-based effectiveness measures and optimization

  • Authors:
  • Stephen Robertson;Hugo Zaragoza

  • Affiliations:
  • Microsoft Research, Cambridge, UK CB3 0FB;Yahoo! Research, Barcelona, Spain 08003

  • Venue:
  • Information Retrieval
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Many current retrieval models and scoring functions contain free parameters which need to be set--ideally, optimized. The process of optimization normally involves some training corpus of the usual document-query-relevance judgement type, and some choice of measure that is to be optimized. The paper proposes a way to think about the process of exploring the space of parameter values, and how moving around in this space might be expected to affect different measures. One result, concerning local optima, is demonstrated for a range of rank-based evaluation measures.