Long-Term Learning for Web Search Engines
PKDD '02 Proceedings of the 6th European Conference on Principles of Data Mining and Knowledge Discovery
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
Engineering a multi-purpose test collection for web retrieval experiments
Information Processing and Management: an International Journal
Evaluating implicit measures to improve web search
ACM Transactions on Information Systems (TOIS)
Accurately interpreting clickthrough data as implicit feedback
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
When will information retrieval be "good enough"?
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Query chains: learning to rank from implicit feedback
Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing)
SIGIR '06 The 29th Annual International SIGIR Conference
Learning user interaction models for predicting web search result preferences
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
User performance versus precision measures for simple search tasks
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Improving web search ranking by incorporating user behavior information
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search
ACM Transactions on Information Systems (TOIS)
The 30th Annual International SIGIR Conference
Fast generation of result snippets in web search
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Random walks on the click graph
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Active exploration for learning rankings from clickthrough data
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Matching task profiles and user needs in personalized web search
Proceedings of the 17th ACM conference on Information and knowledge management
Are Clickthroughs Useful for Image Labelling?
WI-IAT '09 Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 01
Classifying Images with Image and Text Search Clickthrough Data
AMT '09 Proceedings of the 5th International Conference on Active Media Technology
Learning to re-rank: query-dependent image re-ranking using click data
Proceedings of the 20th international conference on World wide web
Implicit association via crowd-sourced coselection
Proceedings of the 22nd ACM conference on Hypertext and hypermedia
Reliability and effectiveness of clickthrough data for automatic image annotation
Multimedia Tools and Applications
A probabilistic method for inferring preferences from clicks
Proceedings of the 20th ACM international conference on Information and knowledge management
Automatically adapting the context of an intranet query
FDIA'08 Proceedings of the 2nd BCS IRSG conference on Future Directions in Information Access
On caption bias in interleaving experiments
Proceedings of the 21st ACM international conference on Information and knowledge management
Evaluating implicit judgments from image search clickthrough data
Journal of the American Society for Information Science and Technology
Discovering semantic associations from web search interactions
Proceedings of the 24th ACM Conference on Hypertext and Social Media
Fidelity, Soundness, and Efficiency of Interleaved Comparison Methods
ACM Transactions on Information Systems (TOIS)
Relative confidence sampling for efficient on-line ranker evaluation
Proceedings of the 7th ACM international conference on Web search and data mining
| Hi-index | 0.00 |
Clickthrough data has been the subject of increasing popularity as an implicit indicator of user feedback. Previous analysis has suggested that user click behaviour is subject to a quality bias--that is, users click at different rank positions when viewing effective search results than when viewing less effective search results. Based on this observation, it should be possible to use click data to infer the quality of the underlying search system. In this paper we carry out a user study to systematically investigate how click behaviour changes for different levels of search system effectiveness as measured by information retrieval performance metrics. Our results show that click behaviour does not vary systematically with the quality of search results. However, click behaviour does vary significantly between individual users, and between search topics. This suggests that using direct click behaviour--click rank and click frequency--to infer the quality of the underlying search system is problematic. Further analysis of our user click data indicates that the correspondence between clicks in a search result list and subsequent confirmation that the clicked resource is actually relevant is low. Using clicks as an implicit indication of relevance should therefore be done with caution.