Crowdsourcing for search evaluation

Authors:
Vitor R. Carvalho;Matthew Lease;Emine Yilmaz
Affiliations:
Microsoft Corporation, Redmond, WA;School of Information, University of Texas at Austin, Austin, TX;Microsoft Research Cambridge, Cambridge, UK
Venue:
ACM SIGIR Forum
Year:
2011

Citing 4
Cited 8

The Cranfield tests on index language devices

Readings in information retrieval
Minimal test collections for retrieval evaluation

SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
A statistical method for system evaluation using incomplete judgments

SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
A simple and efficient sampling method for estimating AP and NDCG

Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

Crowdsourcing for search and data mining

ACM SIGIR Forum
Crowdsourcing for information retrieval

ACM SIGIR Forum
Quality through flow and immersion: gamifying crowdsourced relevance assessments

SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
PROMISE retreat report prospects and opportunities for information access evaluation

ACM SIGIR Forum
Comparing crowd-based, game-based, and machine-based approaches in initial query and query refinement tasks

ECIR'13 Proceedings of the 35th European conference on Advances in Information Retrieval
Crowdsourcing for information retrieval: introduction to the special issue

Information Retrieval
Crowdsourcing and the crisis-affected community

Information Retrieval
Evaluation in Music Information Retrieval

Journal of Intelligent Information Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

The Crowdsourcing for Search Evaluation Workshop (CSE 2010) was held on July 23, 2010 in Geneva, Switzerland, in conjunction with the 33rd Annual ACM SIGIR Conference. The workshop addressed the latest advances in theory and empirical methods in crowdsourcing for search evaluation, as well as novel applications of crowdsourcing for evaluating search systems. Three invited talks were presented, along with seven refereed papers. Proceedings from the workshop, along with presentation slides, have been made available online.