Presentation bias is significant in determining user preference for search results—A user study

Authors:
Judit Bar-Ilan;Kevin Keenoy;Mark Levene;Eti Yaari
Affiliations:
Department of Information Science, Bar-Ilan University, Ramat Gan, 52900, Israel;School of Computer Science and Information Systems, Birkbeck, University of London, Malet Street, London, United Kingdom;School of Computer Science and Information Systems, Birkbeck, University of London, Malet Street, London, United Kingdom;Department of Information Science, Bar-Ilan University, Ramat Gan, 52900, Israel
Venue:
Journal of the American Society for Information Science and Technology
Year:
2009

Citing 0
Cited 9

Topic-specific analysis of search queries

Proceedings of the 2009 workshop on Web Search Click Data
Economic and financial factors for the adoption and visibility effects of Web accessibility: The case of European banks

Journal of the American Society for Information Science and Technology
Creating and visualizing fuzzy document classification

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
The impact of task phrasing on the choice of search keywords and on the search process and success

Journal of the American Society for Information Science and Technology
Incorporating vertical results into search click models

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Pairwise learning in recommendation: experiments with community recommendation on linkedin

Proceedings of the 7th ACM conference on Recommender systems
Search engine accessibility for low-literate users

HCI'13 Proceedings of the 15th international conference on Human-Computer Interaction: users and contexts of use - Volume Part III
Bias in algorithmic filtering and personalization

Ethics and Information Technology
Acessibilidade de recursos em uma interface de motor de busca com foco em usuários com baixo letramento

Proceedings of the 12th Brazilian Symposium on Human Factors in Computing Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe the results of an experiment designed to study user preferences for different orderings of search results from three major search engines. In the experiment, 65 users were asked to choose the best ordering from two different orderings of the same set of search results: Each pair consisted of the search engine's original top-10 ordering and a synthetic ordering created from the same top-10 results retrieved by the search engine. This process was repeated for 12 queries and nine different synthetic orderings. The results show that there is a slight overall preference for the search engines' original orderings, but the preference is rarely significant. Users' choice of the “best” result from each of the different orderings indicates that placement on the page (i.e., whether the result appears near the top) is the most important factor used in determining the quality of the result, not the actual content displayed in the top-10 snippets. In addition to the placement bias, we detected a small bias due to the reputation of the sites appearing in the search results. © 2009 Wiley Periodicals, Inc.