Evaluating relevance feedback algorithms for searching on small displays

  • Authors:
  • Vishwa Vinay;Ingemar J. Cox;Natasa Milic-Frayling;Ken Wood

  • Affiliations:
  • Department of Computer Science, University College London, UK;Department of Computer Science, University College London, UK;Microsoft Research Ltd, Cambridge, UK;Microsoft Research Ltd, Cambridge, UK

  • Venue:
  • ECIR'05 Proceedings of the 27th European conference on Advances in Information Retrieval Research
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Searching online information resources using mobile devices is affected by displays on which only a small fraction of the set of ranked documents can be displayed. In this paper, we ask whether the search effort can be reduced, on average, by user feedback indicating a single most relevant document in each display. For small display sizes and limited user actions, we are able to construct a tree representing all possible outcomes. Examination of the tree permits us to compute an upper limit on relevance feedback performance. Three standard feedback algorithms are considered – Rocchio, Robertson/Sparck-Jones and a Bayesian algorithm. Two display strategies are considered, one based on maximizing the immediate information gain and the other on most likely documents. Our results bring out the strengths and weaknesses of the algorithms, and the need for exploratory display strategies with conservative feedback algorithms.