A pragmatic procedure to support the user-centric evaluation of recommender systems

  • Authors:
  • Bart P. Knijnenburg;Martijn C. Willemsen;Alfred Kobsa

  • Affiliations:
  • University of California, Irvine, Irvine, CA, USA;Eindhoven University of Technology, Eindhoven, Netherlands;University of California, Irvine, Irvine, CA, USA

  • Venue:
  • Proceedings of the fifth ACM conference on Recommender systems
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

As recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also "online" with test users to ensure good user experience. The user evaluation of recommenders is however complex and resource-consuming. We introduce a pragmatic procedure to evaluate recommender systems for experience products with test users, within industry constraints on time and budget. Researchers and practitioners can employ our approach to gain a comprehensive understanding of the user experience with their systems.