Evaluating collaborative filtering recommender systems
ACM Transactions on Information Systems (TOIS)
Being accurate is not enough: how accuracy metrics have hurt recommender systems
CHI '06 Extended Abstracts on Human Factors in Computing Systems
Recommender systems and their impact on sales diversity
Proceedings of the 8th ACM conference on Electronic commerce
The value of personalised recommender systems to e-business: a case study
Proceedings of the 2008 ACM conference on Recommender systems
Seven pitfalls to avoid when running controlled experiments on the web
Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
A case study on the effectiveness of recommendations in the mobile internet
Proceedings of the third ACM conference on Recommender systems
Do clicks measure recommendation relevancy?: an empirical user study
Proceedings of the fourth ACM conference on Recommender systems
Beyond accuracy: evaluating recommender systems by coverage and serendipity
Proceedings of the fourth ACM conference on Recommender systems
Rank and relevance in novelty and diversity metrics for recommender systems
Proceedings of the fifth ACM conference on Recommender systems
A user-centric evaluation framework for recommender systems
Proceedings of the fifth ACM conference on Recommender systems
| Hi-index | 0.00 |
While much of the recommender systems literature has focused on the off-line evaluation of prediction performance, a few case studies using online controlled experiments that assess the performance of business indicators are available. In this article, we describe the methods and results of an ongoing investigation conducted on the business value impact of personalized recommendations on three different portals of Nova Pontocom, the second largest Latin American online retailer. An on-line controlled experiment (A/B testing), conducted for one month and covering 600,000 distinct users, statistically points out to a general revenue increase in the order of 8-20%. In addition, other consumer behavior metrics such as the number of page views and the more diverse distribution of sales among the products catalog also support the positive impact of personalized recommendations in terms of business value.