Use cases as a component of information access evaluation

  • Authors:

  • Jussi Karlgren;Anni Järvelin;Gunnar Eriksson;Preben Hansen

  • Affiliations:

  • Swedish Institute of Computer Science, Stockholm, Sweden;Swedish Institute of Computer Science, Stockholm, Sweden;Swedish Institute of Computer Science, Stockholm, Sweden;Swedish Institute of Computer Science, Stockholm, Sweden

  • Venue:
  • Proceedings of the 2011 workshop on Data infrastructurEs for supporting information retrieval evaluation
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Information access research and development, and information retrieval especially, is based on quantitative and systematic benchmarking. Benchmarking of a computational mechanism is always based on some set of assumptions on how a system with the mechanism under consideration will provide value for its users in concrete situations and those assumptions need to be validated somehow. The valuable effort put into those validation studies is seldom useful for other research or system development projects. This paper argues that use cases for information access can be written to give explicit pointers towards benchmarking mechanisms and that if use cases and hypotheses about user preferences, goals, expectation and satisfaction are made explicit in the design of research systems, they will can more conveniently be validated or disproven -- which in turn makes the results emanating from research efforts more relevant for industrial partners, more sustainable for future research and more portable across projects and studies.