Spying out accurate user preferences for search engine adaptation

  • Authors:
  • Lin Deng;Wilfred Ng;Xiaoyong Chai;Dik-Lun Lee

  • Affiliations:
  • Department of Computer Science, Hong Kong University of Science and Technology;Department of Computer Science, Hong Kong University of Science and Technology;Department of Computer Science, Hong Kong University of Science and Technology;Department of Computer Science, Hong Kong University of Science and Technology

  • Venue:
  • WebKDD'04 Proceedings of the 6th international conference on Knowledge Discovery on the Web: advances in Web Mining and Web Usage Analysis
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Most existing search engines employ static ranking algorithms that do not adapt to the specific needs of users. Recently, some researchers have studied the use of clickthrough data to adapt a search engine’s ranking function. Clickthrough data indicate for each query the results that are clicked by users. As a kind of implicit relevance feedback information, clickthrough data can easily be collected by a search engine. However, clickthrough data is sparse and incomplete, thus, it is a challenge to discover accurate user preferences from it. In this paper, we propose a novel algorithm called “Spy Naïve Bayes” (SpyNB) to identify user preferences generated from clickthrough data. First, we treat the result items clicked by the users as sure positive examples and those not clicked by the users as unlabelled data. Then, we plant the sure positive examples (the spies) into the unlabelled set of result items and apply a naïve Bayes classification to generate the reliable negative examples. These positive and negative examples allow us to discover more accurate user’s preferences. Finally, we employ the SpyNB algorithm with a ranking SVM optimizer to build an adaptive metasearch engine. Our experimental results show that, compared with the original ranking, SpyNB can significantly improve the average ranks of users’ click by 20%.