Relative effect of spam and irrelevant documents on user interaction with search engines

Authors:
Timothy Jones;David Hawking;Paul Thomas;Ramesh Sankaranarayana
Affiliations:
Australian National University, Canberra, Australia;Funnelback Pty Ltd. and Australian National University, Canberra, Australia;CSIRO, Canberra, Australia;Australian National University, Canberra, Australia
Venue:
Proceedings of the 20th ACM international conference on Information and knowledge management
Year:
2011

Citing 6
Cited 2

Identifying web spam with user behavior analysis

AIRWeb '08 Proceedings of the 4th international workshop on Adversarial information retrieval on the web
Web Spam Identification with User Browsing Graph

AIRS '09 Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology
A review of factors influencing user satisfaction in information retrieval

Journal of the American Society for Information Science and Technology
Helping identify when users find useful documents: examination of query reformulation intervals

Proceedings of the third symposium on Information interaction in context
Adversarial Web Search

Foundations and Trends in Information Retrieval
What deliberately degrading search quality tells us about discount functions

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval

Users versus models: what observation tells us about effectiveness metrics

Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Choices in batch information retrieval evaluation

Proceedings of the 18th Australasian Document Computing Symposium

Quantified Score

Hi-index	0.01

Visualization

Abstract

Meaningful evaluation of web search must take account of spam. Here we conduct a user experiment to investigate whether satisfaction with search engine result pages as a whole is harmed more by spam or by irrelevant documents. On some measures, search result pages are differentially harmed by the insertion of spam and irrelevant documents. Additionally we find that when users are given two documents of equal utility, the one with the lower spam score will be preferred; a result page without any spam documents will be preferred to one with spam; and an irrelevant document high in a result list is surprisingly more damaging to user satisfaction than a spam document. We conclude that web ranking and evaluation should consider both utility (relevance) and "spamminess" of documents.