Explaining collaborative filtering recommendations
CSCW '00 Proceedings of the 2000 ACM conference on Computer supported cooperative work
Item-based collaborative filtering recommendation algorithms
Proceedings of the 10th international conference on World Wide Web
Amazon.com Recommendations: Item-to-Item Collaborative Filtering
IEEE Internet Computing
A collaborative filtering algorithm and evaluation metric that accurately model the user experience
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Scalable collaborative filtering using cluster-based smoothing
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
An introduction to ROC analysis
Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
A Tutorial on Conformal Prediction
The Journal of Machine Learning Research
Estimating the Confidence Interval for Prediction Errors of Support Vector Machine Classifiers
The Journal of Machine Learning Research
Reliability Assessment of Ensemble Classifiers: Application in Mammography
IWDM '08 Proceedings of the 9th international workshop on Digital Mammography
Mean-Variance Analysis: A New Document Ranking Theory in Information Retrieval
ECIR '09 Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval
A Collaborative Filtering Algorithm Based on Variance Analysis of Attributes-Value Preference
ICMECG '09 Proceedings of the 2009 International Conference on Management of e-Commerce and e-Government
A survey of collaborative filtering techniques
Advances in Artificial Intelligence
A Survey of Accuracy Evaluation Metrics of Recommendation Tasks
The Journal of Machine Learning Research
Back to the roots: mean-variance analysis of relevance estimations
ECIR'11 Proceedings of the 33rd European conference on Advances in information retrieval
OrdRec: an ordinal model for predicting personalized item rating distributions
Proceedings of the fifth ACM conference on Recommender systems
Empirical analysis of predictive algorithms for collaborative filtering
UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Expert Systems with Applications: An International Journal
Robustness analysis of privacy-preserving model-based recommendation schemes
Expert Systems with Applications: An International Journal
| Hi-index | 12.05 |
Collaborative filtering algorithms predict a rating for an item based on the user's previous ratings for other items as well as ratings of other users. This approach has gained notable popularity both in academic research and in commercial applications. One aspect of collaborative filtering systems that received interest, but little systematic analysis, is confidence of the rating predictions by collaborative filtering algorithms. In this paper, I address this issue. Specifically: (1) I offer a discussion on the definition of confidence, (2) I propose a method for evaluating performance of confidence estimation algorithms, and (3) I evaluate six different confidence estimation algorithms. Three of those algorithms are introduced in this paper and three have been previously suggested for this purpose. The comparative experimental evaluation demonstrates that two of the algorithms proposed in this study: one using resampling of available ratings and one using noise injection to the available ratings provide the best performance in terms of separation between predictions of high and low confidence. The algorithms that use only the number of ratings available for the user of interest or for the item of interest turned out to be of limited use for confidence estimation.