Workshop on reproducibility and replication in recommender systems evaluation: RepSys

Authors:
Alejandro Bellogin;Pablo Castells;Alan Said;Domonkos Tikk
Affiliations:
Centrum Wiskunde & Informatica, Amsterdam, Netherlands;Universidad Autónoma de Madrid, Madrid, Spain;Centrum Wiskunde & Informatica, Amsterdam, Netherlands;Gravity R&D, Budapest, Hungary
Venue:
Proceedings of the 7th ACM conference on Recommender systems
Year:
2013

Citing 4
Cited 0

SIGIR06 workshop report: Open Source Information Retrieval systems (OSIR06)

ACM SIGIR Forum
The Need for Open Source Software in Machine Learning

The Journal of Machine Learning Research
RSSE 2010: Second International Workshop on Recommendation Systems for Software Engineering

Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2
Open source information retrieval: a report on the SIGIR 2012 workshop

ACM SIGIR Forum

Quantified Score

Hi-index	0.00

Visualization

Abstract

Experiment replication and reproduction are key requirements for empirical research methodology, and an important open issue in the field of Recommender Systems. When an experiment is repeated by a different researcher and exactly the same result is obtained, we can say the experiment has been replicated. When the results are not exactly the same but the conclusions are compatible with the prior ones, we have a reproduction of the experiment. Reproducibility and replication involve recommendation algorithm implementations, experimental protocols, and evaluation metrics. While the problem of reproducibility and replication has been recognized in the Recommender Systems community, the need for a clear solution remains largely unmet, which motivates the present workshop.