IR evaluation methods for retrieving highly relevant documents
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
ACM SIGIR Forum
Implicit feedback for inferring user preference: a bibliography
ACM SIGIR Forum
An efficient boosting algorithm for combining preferences
The Journal of Machine Learning Research
Evaluating implicit measures to improve web search
ACM Transactions on Information Systems (TOIS)
Accurately interpreting clickthrough data as implicit feedback
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Query chains: learning to rank from implicit feedback
Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Learning to rank using gradient descent
ICML '05 Proceedings of the 22nd international conference on Machine learning
Learning user interaction models for predicting web search result preferences
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Improving web search ranking by incorporating user behavior information
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Adapting ranking SVM to document retrieval
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search
ACM Transactions on Information Systems (TOIS)
A large-scale evaluation and analysis of personalized search strategies
Proceedings of the 16th international conference on World Wide Web
A regression framework for learning ranking functions using relative relevance judgments
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Learning to rank for information retrieval (LR4IR 2007)
ACM SIGIR Forum
Minimally invasive randomization for collecting unbiased preferences from clickthrough logs
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Proceedings of the Second ACM International Conference on Web Search and Data Mining
Analyzing and evaluating query reformulation strategies in web search logs
Proceedings of the 18th ACM conference on Information and knowledge management
Improving web page classification by label-propagation over click graphs
Proceedings of the 18th ACM conference on Information and knowledge management
Feature engineering on event-centric surrogate documents to improve search results
Proceedings of the 18th ACM conference on Information and knowledge management
Exploring relevance for clicks
Proceedings of the 18th ACM conference on Information and knowledge management
Exploiting bilingual information to improve web search
ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2
Automatic Search Engine Performance Evaluation with the Wisdom of Crowds
AIRS '09 Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology
Study on the Click Context of Web Search Users for Reliability Analysis
AIRS '09 Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology
Large-scale bot detection for search engines
Proceedings of the 19th international conference on World wide web
Context-sensitive document ranking
Journal of Computer Science and Technology
Learning to rank with multiple objective functions
Proceedings of the 20th international conference on World wide web
Efficiently collecting relevance information from clickthroughs for web retrieval system evaluation
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Empirical Study on Rare Query Characteristics
WI-IAT '11 Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Reranking search results for sparse queries
Proceedings of the 20th ACM international conference on Information and knowledge management
Survey on web spam detection: principles and algorithms
ACM SIGKDD Explorations Newsletter
Deriving query intents from web search engine queries
Journal of the American Society for Information Science and Technology
On caption bias in interleaving experiments
Proceedings of the 21st ACM international conference on Information and knowledge management
Incorporating social anchors for ad hoc retrieval
Proceedings of the 10th Conference on Open Research Areas in Information Retrieval
Mining search and browse logs for web search: A Survey
ACM Transactions on Intelligent Systems and Technology (TIST) - Survey papers, special sections on the semantic adaptive social web, intelligent systems for health informatics, regular papers
Hi-index | 0.00 |
Learning-to-rank algorithms, which can automatically adapt ranking functions in web search, require a large volume of training data. A traditional way of generating training examples is to employ human experts to judge the relevance of documents. Unfortunately, it is difficult, time-consuming and costly. In this paper, we study the problem of exploiting click-through data for learning web search rankings that can be collected at much lower cost. We extract pairwise relevance preferences from a large-scale aggregated click-through dataset, compare these preferences with explicit human judgments, and use them as training examples to learn ranking functions. We find click-through data are useful and effective in learning ranking functions. A straightforward use of aggregated click-through data can outperform human judgments. We demonstrate that the strategies are only slightly affected by fraudulent clicks. We also reveal that the pairs which are very reliable, e.g., the pairs consisting of documents with large click frequency differences, are not sufficient for learning.