Report on the SIGIR 2013 workshop on modeling user behavior for information retrieval evaluation (MUBE 2013)

Authors:
Charles L.A. Clarke;Luanne Freund;Mark D. Smucker;Emine Yilmaz
Affiliations:
University of Waterloo, Canada;University of British Columbia, Canada;University of Waterloo, Canada;University College London, UK
Venue:
ACM SIGIR Forum
Year:
2013

Citing 38
Cited 0

Incremental relevance feedback

SIGIR '92 Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval
Time, relevance and interaction modelling for information retrieval

Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval
Using information scent to model user information needs and actions and the Web

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Cumulated gain-based evaluation of IR techniques

ACM Transactions on Information Systems (TOIS)
Re-examining the potential effectiveness of interactive query expansion

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Modeling User Search Behavior

LA-WEB '05 Proceedings of the Third Latin American Web Congress
Predictive modeling of first-click behavior in web-search

Proceedings of the 15th international conference on World Wide Web
Find-similar: similarity browsing as a search tool

SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
User simulations for evaluating answers to question series

Information Processing and Management: an International Journal
How do users find things with PubMed?: towards automatic utility evaluation with user simulations

Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
A new interpretation of average precision

Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Rank-biased precision for measurement of retrieval effectiveness

ACM Transactions on Information Systems (TOIS)
Modeling actions of PubMed users with n-gram language models

Information Retrieval
Including summaries in system evaluation

Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Modeling Expected Utility of Multi-session Information Distillation

ICTIR '09 Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory
Expected reciprocal rank for graded relevance

Proceedings of the 18th ACM conference on Information and knowledge management
Usage based effectiveness measures: monitoring application performance in information retrieval

Proceedings of the 18th ACM conference on Information and knowledge management
Test Collection-Based IR Evaluation Needs Extension toward Sessions --- A Case of Extremely Short Queries

AIRS '09 Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology
Beyond DCG: user behavior as a predictor of a successful search

Proceedings of the third ACM international conference on Web search and data mining
Click-based evidence for decaying weight distributions in search effectiveness metrics

Information Retrieval
A user behavior model for average precision and its generalization to graded judgments

Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
The anatomy of a click: modeling user behavior on web information systems

CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Expected browsing utility for web search evaluation

CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Expected reading effort in focused retrieval evaluation

Information Retrieval
Report on the SIGIR 2010 workshop on the simulation of interaction

ACM SIGIR Forum
The economics in interactive information retrieval

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Find it if you can: a game for modeling different types of web search success using interaction data

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
System effectiveness, user models, and user utility: a conceptual framework for investigation

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Evaluating multi-query sessions

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Discounted cumulative gain and user decision models

SPIRE'11 Proceedings of the 18th international conference on String processing and information retrieval
Simulating simple user behavior for system effectiveness evaluation

Proceedings of the 20th ACM international conference on Information and knowledge management
Time-based calibration of effectiveness measures

SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Time drives interaction: simulating sessions in diverse searching environments

SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Modeling user variance in time-biased gain

Proceedings of the Symposium on Human-Computer Interaction and Information Retrieval
Incorporating variability in user behavior into systems based evaluation

Proceedings of the 21st ACM international conference on Information and knowledge management
Summaries, ranked retrieval and sessions: a unified framework for information access evaluation

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
How query cost affects search behavior

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Click model-based information retrieval metrics

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

The SIGIR 2013 Workshop on Modeling User Behavior of Information Retrieval Evaluation brought together researchers interested in improving Cranfield-style evaluation of information retrieval through the modeling of user behavior. The workshop included two invited talks, ten short paper presentations, and breakout groups. Workshop participants brainstormed research questions of interest and formed breakout groups to explore these questions in greater depth. In addition to summarizing the invited talks and presentations, this report details the results of the breakout groups, which provide a set of research directions for the improvement of information retrieval evaluation.