Relevance criteria for e-commerce: a crowdsourcing-based experimental analysis

Authors:
Omar Alonso;Stefano Mizzaro
Affiliations:
A9.com, Palo Alto, CA, USA;University of Udine, Udine, IA, Italy
Venue:
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Year:
2009

Citing 6
Cited 5

Relevance: the whole history

Journal of the American Society for Information Science - Special topic issue on the history of documentation and information science: part II
Users' criteria for relevance evaluation: a cross-situational comparison

Information Processing and Management: an International Journal
Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance

Journal of the American Society for Information Science and Technology
Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance

Journal of the American Society for Information Science and Technology
Crowdsourcing user studies with Mechanical Turk

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Crowdsourcing for relevance evaluation

ACM SIGIR Forum

IR Evaluation without a Common Set of Topics

ICTIR '09 Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory
Putting the crowd to work in a knowledge-based factory

Advanced Engineering Informatics
A user-tunable approach to marketplace search

Proceedings of the 20th international conference companion on World wide web
An examination of two delivery modes for interactive search system experiments: remote and laboratory

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Using crowdsourcing for TREC relevance assessment

Information Processing and Management: an International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

We discuss the concept of relevance criteria in the context of e-Commerce search. A vast body of research literature describes the beyond-topical criteria used to determine the relevance of the document to the need. We argue that in an e-Commerce scenario there are some differences, and novel and different criteria can be used to determine relevance. We experimentally validate this hypothesis by means of Amazon Mechanical Turk using a crowdsourcing approach.