An MDP-based recommender system

Authors:
Guy Shani;Ronen I. Brafman;David Heckerman
Affiliations:
Department of Computer Science, Ben-Gurion University, Beer-Sheva, Israel;Department of Computer Science, Ben-Gurion University, Beer-Sheva, Israel;Microsoft Research, Redmond, WA
Venue:
UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
Year:
2002

Citing 7
Cited 10

Recommender systems in e-commerce

Proceedings of the 1st ACM conference on Electronic commerce
Analysis of recommendation algorithms for e-commerce

Proceedings of the 2nd ACM conference on Electronic commerce
A prediction system for multimedia pre-fetching in Internet

MULTIMEDIA '00 Proceedings of the eighth ACM international conference on Multimedia
Using Temporal Data for Making Recommendations

UAI '01 Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence
Reinforcement learning with selective perception and hidden state

Reinforcement learning with selective perception and hidden state
Dependency networks for inference, collaborative filtering, and data visualization

The Journal of Machine Learning Research
Empirical analysis of predictive algorithms for collaborative filtering

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence

Maximum entropy for collaborative filtering

UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Collaborative Filtering with Maximum Entropy

IEEE Intelligent Systems
Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions

IEEE Transactions on Knowledge and Data Engineering
Google news personalization: scalable online collaborative filtering

Proceedings of the 16th international conference on World Wide Web
Advice generation from observed execution: abstract Markov decision process learning

AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Recsplorer: recommendation algorithms based on precedence mining

Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
Decision-Theoretic planning meets user requirements: enhancements and studies of an intelligent shopping guide

PERVASIVE'05 Proceedings of the Third international conference on Pervasive Computing
RecStore: an extensible and adaptive framework for online recommender queries inside the database engine

Proceedings of the 15th International Conference on Extending Database Technology
Assessing the appropriateness of using markov decision processes for RF spectrum management

Proceedings of the 16th ACM international conference on Modeling, analysis & simulation of wireless and mobile systems
Mining novelty-seeking trait across heterogeneous domains

Proceedings of the 23rd international conference on World wide web

Quantified Score

Hi-index	0.00

Visualization

Abstract

Typical Recommender systems adopt a static view of the recommendation process and treat it as a prediction problem. We argue that it is more appropriate to view the problem of generating recommendations as a sequential decision problem and, consequently, that Markov decision processes (MDP) provide a more appropriate model for Recommender systems. MDPs introduce two benefits: they take into account the long-term effects of each recommendation, and they take into account the expected value of each recommendation. To succeed in practice, an MDP-based Recommender system must employ a strong initial model; and the bulk of this paper is concerned with the generation of such a model. In particular, we suggest the use of an n-gram predictive model for generating the initial MDP. Our n-gram model induces a Markovchain model of user behavior whose predictive accuracy is greater than that of existing predictive models. We describe our predictive model in detail and evaluate its performance on real data. In addition, we show how the model can be used in an MDP-based Recommender system.