Evaluating collaborative filtering recommender systems
ACM Transactions on Information Systems (TOIS)
A collaborative filtering algorithm and evaluation metric that accurately model the user experience
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Being accurate is not enough: how accuracy metrics have hurt recommender systems
CHI '06 Extended Abstracts on Human Factors in Computing Systems
Factorization meets the neighborhood: a multifaceted collaborative filtering model
Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
kNN CF: a temporal social network
Proceedings of the 2008 ACM conference on Recommender systems
Performance of recommender algorithms on top-n recommendation tasks
Proceedings of the fourth ACM conference on Recommender systems
Optimizing multiple objectives in collaborative filtering
Proceedings of the fourth ACM conference on Recommender systems
Goal-driven collaborative filtering – a directional error based approach
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Using past-prediction accuracy in recommender systems
Information Sciences: an International Journal
ACM Transactions on Interactive Intelligent Systems (TiiS)
User effort vs. accuracy in rating-based elicitation
Proceedings of the sixth ACM conference on Recommender systems
Using graph partitioning techniques for neighbour selection in user-based collaborative filtering
Proceedings of the sixth ACM conference on Recommender systems
A comparative study of heterogeneous item recommendations in social systems
Information Sciences: an International Journal
Expert Systems with Applications: An International Journal
Relevance-based language modelling for recommender systems
Information Processing and Management: an International Journal
Probabilistic group recommendation via information matching
Proceedings of the 22nd international conference on World Wide Web
Exploiting the diversity of user preferences for recommendation
Proceedings of the 10th Conference on Open Research Areas in Information Retrieval
Probabilistic collaborative filtering with negative cross entropy
Proceedings of the 7th ACM conference on Recommender systems
Bridging memory-based collaborative filtering and text retrieval
Information Retrieval
Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols
User Modeling and User-Adapted Interaction
| Hi-index | 0.00 |
There is considerable methodological divergence in the way precision-oriented metrics are being applied in the Recommender Systems field, and as a consequence, the results reported in different studies are difficult to put in context and compare. We aim to identify the involved methodological design alternatives, and their effect on the resulting measurements, with a view to assessing their suitability, advantages, and potential shortcomings. We compare five experimental methodologies, broadly covering the variants reported in the literature. In our experiments with three state-of-the-art recommenders, four of the evaluation methodologies are consistent with each other and differ from error metrics, in terms of the comparative recommenders' performance measurements. The other procedure aligns with RMSE, but shows a heavy bias towards known relevant items, considerably overestimating performance.