The evaluation of adaptive and personalised information retrieval systems: a review

  • Authors:
  • Catherine Mulwa;Seamus Lawless;Mary Sharp;Vincent Wade

  • Affiliations:
  • Centre for Next Generation Localisation, Knowledge and Data Engineering Group, School of Computer Science and Statistics, Trinity College, Dublin, Ireland;Centre for Next Generation Localisation, Knowledge and Data Engineering Group, School of Computer Science and Statistics, Trinity College, Dublin, Ireland;Centre for Next Generation Localisation, Knowledge and Data Engineering Group, School of Computer Science and Statistics, Trinity College, Dublin, Ireland;Centre for Next Generation Localisation, Knowledge and Data Engineering Group, School of Computer Science and Statistics, Trinity College, Dublin, Ireland

  • Venue:
  • International Journal of Knowledge and Web Intelligence
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

A current problem with the research of adaptive systems is the inconsistency of evaluation applied to the adaptive systems. However, evaluating an adaptive system is a difficult task due to the complexity of such systems. Evaluators need to ensure correct evaluation methods and measurement metrics are used. This paper reviews a variety of evaluation techniques applied in adaptive and user-adaptive systems. More specifically, it focuses on the user-centred evaluation of adaptive systems such as personalised recommender systems and adaptive information retrieval systems. The review tackles the question of "ï戮聵How have user-centred evaluations of adaptive and user-adaptive systems been conducted and how can these evaluation practices be improved?' Based on the analysed results of the: (a) evaluation approaches, (b) user-centred evaluation techniques, and (c) evaluation metrics, we propose an evaluation framework for end-user experience in evaluating adaptive systems (EFEx).