Experiences evaluating personal metasearch

  • Authors:
  • Paul Thomas;David Hawking

  • Affiliations:
  • CSIRO ICT Centre, Canberra, Australia;Funnelback Pty Ltd, Canberra, Australia

  • Venue:
  • Proceedings of the second international symposium on Information interaction in context
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Many current evaluation techniques for information retrieval, such as test collections and simulations, are difficult to apply in situations where queries and preferred results are context-dependent. This is particularly true in personal metasearch applications, which provide a person with unified search access to all their usual online sources. A recently-proposed technique, based on presenting two or more search results sets in a single comparison interface, offers an alternative. We have embedded this technique in a working personal metasearch tool which we have distributed to volunteers. Initial experiments with server selection suggest that the technique is accepted by users, can operate over diverse and unarticulated contexts, and that the data it provides can provide a useful comparison to that from test collections. Further experimentation with the technique is continuing.