Recommending and evaluating choices in a virtual community of use
CHI '95 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Evaluating collaborative filtering recommender systems
ACM Transactions on Information Systems (TOIS)
Lessons from the Netflix prize challenge
ACM SIGKDD Explorations Newsletter - Special issue on visual analytics
I Like It... I Like It Not: Evaluating User Ratings Noise in Recommender Systems
UMAP '09 Proceedings of the 17th International Conference on User Modeling, Adaptation, and Personalization: formerly UM and AH
Rate it again: increasing recommendation accuracy by user re-rating
Proceedings of the third ACM conference on Recommender systems
Hi-index | 0.00 |
Recommender systems are commonly evaluated by trying to predict known, withheld, ratings for a set of users. Measures such as the Root-Mean-Square Error are used to estimate the quality of the recommender algorithms. This process does however not acknowledge the inherent rating inconsistencies of users. In this paper we present the first results from a noise measurement user study for estimating the magic barrier of recommender systems conducted on a commercial movie recommendation community. The magic barrier is the expected squared error of the optimal recommendation algorithm, or, the lowest error we can expect from any recommendation algorithm. Our results show that the barrier can be estimated by collecting the opinions of users on already rated items.