Multi-Dimensional Evaluation of Information Retrieval Results

  • Authors:
  • Xiangzhu Gao;San Murugesan;Bruce Lo

  • Affiliations:
  • Southern Cross University, Australia;Southern Cross University, Australia;University of Wisconsin-Eau Claire

  • Venue:
  • WI '04 Proceedings of the 2004 IEEE/WIC/ACM International Conference on Web Intelligence
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Evaluation of information retrieval (IR) results is an important and difficult activity. Available evaluation methods are commonly based on the classical recall and precision measures. While these methods provide an evaluation of average system performance, they are not able to identify influence of other factors such as retrieval task and system user. Detailed information is hidden behind the average and no indication can be obtained from the evaluation for further improvement. Therefore, there is a need of new methods for detailed evaluation. In this paper, we propose a multi-dimensional approach to the evaluation of IR results so that effects of both an IR system and the environment where the system performs on retrieval results can be examined. It aids in the identification of problems of the system and the environment with respect to retrieval effectiveness, and assists in making improvement for IR.