Evaluating collaborative filtering recommender systems
ACM Transactions on Information Systems (TOIS)
TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing)
Being accurate is not enough: how accuracy metrics have hurt recommender systems
CHI '06 Extended Abstracts on Human Factors in Computing Systems
A new approach to evaluating novel recommendations
Proceedings of the 2008 ACM conference on Recommender systems
Temporal diversity in recommender systems
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Performance of recommender algorithms on top-n recommendation tasks
Proceedings of the fourth ACM conference on Recommender systems
Comparative evaluation of recommender system quality
CHI '11 Extended Abstracts on Human Factors in Computing Systems
Rank and relevance in novelty and diversity metrics for recommender systems
Proceedings of the fifth ACM conference on Recommender systems
Workshop on novelty and diversity in recommender systems - DiveRS 2011
Proceedings of the fifth ACM conference on Recommender systems
UCERSTI 2: second workshop on user-centric evaluation of recommender systems and their interfaces
Proceedings of the fifth ACM conference on Recommender systems
A 3D approach to recommender system evaluation
Proceedings of the 2013 conference on Computer supported cooperative work companion
| Hi-index | 0.00 |
Measuring the error in rating prediction has been by far the dominant evaluation methodology in the Recommender Systems literature. Yet there seems to be a general consensus that this criterion alone is far from being enough to assess the practical effectiveness of a recommender system. Information Retrieval metrics have started to be used to evaluate item selection and ranking rather than rating prediction, but considerable divergence remains in the adoption of such metrics by different authors. On the other hand, recommendation utility includes other key dimensions and concerns beyond accuracy, such as novelty and diversity, user engagement, and business performance. While the need for further extension, formalization, clarification and standardization of evaluation methodologies is recognized in the community, this need is still unmet for a large extent. The RUE 2012 workshop sought to identify and better understand the current gaps in recommender system evaluation methodologies, help lay directions for progress in addressing them, and contribute to the consolidation and convergence of experimental methods and practice.