Results and challenges in Web search evaluation
WWW '99 Proceedings of the eighth international conference on World Wide Web
Cumulated gain-based evaluation of IR techniques
ACM Transactions on Information Systems (TOIS)
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
ACM SIGIR Forum
Eye-tracking analysis of user behavior in WWW search
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Evaluating implicit measures to improve web search
ACM Transactions on Information Systems (TOIS)
Query chains: learning to rank from implicit feedback
Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Learning user interaction models for predicting web search result preferences
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Improving web search ranking by incorporating user behavior information
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Click data as implicit relevance feedback in web search
Information Processing and Management: an International Journal
Investigating behavioral variability in web search
Proceedings of the 16th international conference on World Wide Web
Experimental perspectives on learning from imbalanced data
Proceedings of the 24th international conference on Machine learning
How well does result relevance predict session satisfaction?
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Active exploration for learning rankings from clickthrough data
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
A user browsing model to predict search engine click data from past observations.
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
How does clickthrough data reflect retrieval quality?
Proceedings of the 17th ACM conference on Information and knowledge management
Beyond the session timeout: automatic hierarchical segmentation of search topics in query logs
Proceedings of the 17th ACM conference on Information and knowledge management
Efficient multiple-click models in web search
Proceedings of the Second ACM International Conference on Web Search and Data Mining
Cutting-plane training of structural SVMs
Machine Learning
Learning when training data are costly: the effect of class distribution on tree induction
Journal of Artificial Intelligence Research
Exploratory undersampling for class-imbalance learning
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
A model to estimate intrinsic document relevance from the clickthrough logs of a web search engine
Proceedings of the third ACM international conference on Web search and data mining
Beyond DCG: user behavior as a predictor of a successful search
Proceedings of the third ACM international conference on Web search and data mining
Comparing the sensitivity of information retrieval metrics
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Evaluating the effectiveness of search task trails
Proceedings of the 21st international conference on World Wide Web
Proceedings of the 21st international conference on World Wide Web
A semi-supervised approach to modeling web search satisfaction
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Search, interrupted: understanding and predicting search task continuation
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Playing by the rules: mining query associations to predict search performance
Proceedings of the sixth ACM international conference on Web search and data mining
Temporal web dynamics and its application to information retrieval
Proceedings of the sixth ACM international conference on Web search and data mining
Toward self-correcting search engines: using underperforming queries to improve search
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Toward whole-session relevance: exploring intrinsic diversity in web search
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Search engine switching detection based on user personal preferences and behavior patterns
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Mining touch interaction data on mobile devices to predict web search result relevance
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Enhancing personalized search by mining and modeling task behavior
Proceedings of the 22nd international conference on World Wide Web
Personalized models of search satisfaction
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Beyond clicks: query reformulation as a predictor of search satisfaction
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Strategy in action: analyzing online search behavior bymining search strategies
Proceedings of the 7th ACM international conference on Web search and data mining
Modeling dwell time to predict click-level satisfaction
Proceedings of the 7th ACM international conference on Web search and data mining
Hi-index | 0.00 |
Understanding the behavior of satisfied and unsatisfied Web search users is very important for improving users search experience. Collecting labeled data that characterizes search behavior is a very challenging problem. Most of the previous work used a limited amount of data collected in lab studies or annotated by judges lacking information about the actual intent. In this work, we performed a large scale user study where we collected explicit judgments of user satisfaction with the entire search task. Results were analyzed using sequence models that incorporate user behavior to predict whether the user ended up being satisfied with a search or not. We test our metric on millions of queries collected from real Web search traffic and show empirically that user behavior models trained using explicit judgments of user satisfaction outperform several other search quality metrics. The proposed model can also be used to optimize different search engine components. We propose a method that uses task level success prediction to provide a better interpretation of clickthrough data. Clickthough data has been widely used to improve relevance estimation. We use our user satisfaction model to distinguish between clicks that lead to satisfaction and clicks that do not. We show that adding new features derived from this metric allowed us to improve the estimation of document relevance.