An introduction to computational learning theory
An introduction to computational learning theory
Rank aggregation methods for the Web
Proceedings of the 10th international conference on World Wide Web
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
An efficient boosting algorithm for combining preferences
The Journal of Machine Learning Research
Comparing and aggregating rankings with ties
PODS '04 Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Evaluating implicit measures to improve web search
ACM Transactions on Information Systems (TOIS)
Aggregating inconsistent information: ranking and clustering
Proceedings of the thirty-seventh annual ACM symposium on Theory of computing
Accurately interpreting clickthrough data as implicit feedback
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Query chains: learning to rank from implicit feedback
Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Learning to rank using gradient descent
ICML '05 Proceedings of the 22nd international conference on Machine learning
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Algorithms for discovering bucket orders from data
Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
A system for query-specific document summarization
CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search
ACM Transactions on Information Systems (TOIS)
What are you looking for?: an eye-tracking study of information usage in web search
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
An eye tracking study of the effect of target rank on web search
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The influence of caption features on clickthrough patterns in web search
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Random walks on the click graph
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Active exploration for learning rankings from clickthrough data
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
SoftRank: optimizing non-smooth rank metrics
WSDM '08 Proceedings of the 2008 International Conference on Web Search and Data Mining
Discovering bucket orders from full rankings
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
EigenRank: a ranking-oriented approach to collaborative filtering
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
A user browsing model to predict search engine click data from past observations.
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Computer
Are click-through data adequate for learning web search rankings?
Proceedings of the 17th ACM conference on Information and knowledge management
Minimally invasive randomization for collecting unbiased preferences from clickthrough logs
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Click fraud resistant methods for learning click-through rates
WINE'05 Proceedings of the First international conference on Internet and Network Economics
Improving classification accuracy using automatically extracted training data
Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
Study on the Click Context of Web Search Users for Reliability Analysis
AIRS '09 Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology
Improving quality of training data for learning to rank using click-through data
Proceedings of the third ACM international conference on Web search and data mining
Incorporating post-click behaviors into a click model
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Learning dense models of query similarity from user click logs
HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Learning click models via probit bayesian inference
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Detecting duplicate web documents using clickthrough data
Proceedings of the fourth ACM international conference on Web search and data mining
Learning to re-rank: query-dependent image re-ranking using click data
Proceedings of the 20th international conference on World wide web
Large-scale validation and analysis of interleaved search evaluation
ACM Transactions on Information Systems (TOIS)
Proceedings of the fifth ACM international conference on Web search and data mining
Modeling click and relevance relationship for sponsored search
Proceedings of the 22nd international conference on World Wide Web companion
User intent and assessor disagreement in web search evaluation
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
| Hi-index | 0.00 |
The ranking function used by search engines to order results is learned from labeled training data. Each training point is a (query, URL) pair that is labeled by a human judge who assigns a score of Perfect, Excellent, etc., depending on how well the URL matches the query. In this paper, we study whether clicks can be used to automatically generate good labels. Intuitively, documents that are clicked (resp., skipped) in aggregate can indicate relevance (resp., lack of relevance). We give a novel way of transforming clicks into weighted, directed graphs inspired by eye-tracking studies and then devise an objective function for finding cuts in these graphs that induce a good labeling. In its full generality, the problem is NP-hard, but we show that, in the case of two labels, an optimum labeling can be found in linear time. For the more general case, we propose heuristic solutions. Experiments on real click logs show that click-based labels align with the opinion of a panel of judges, especially as the consensus of the panel grows stronger.