Order effect in relevance judgment

Authors:
Yunjie Xu;Dong Wang
Affiliations:
3 Science Drive 2, Singapore, 117543;3 Science Drive 2, Singapore, 117543
Venue:
Journal of the American Society for Information Science and Technology
Year:
2008

Citing 0
Cited 4

Addressing gaps in knowledge while reading

Journal of the American Society for Information Science and Technology
Design factors affecting relevance judgment behaviour in the context of metadata surrogates

Journal of Information Science
Content-based image retrieval approach for biometric security using colour, texture and shape features controlled by fuzzy heuristics

Journal of Computer and System Sciences
Enhancing Mobile Advertising Effectiveness in Turkey through Peer Influence

Journal of Electronic Commerce in Organizations

Quantified Score

Hi-index	0.00

Visualization

Abstract

The order effect of relevance judgment refers to the different relevance perceptions of a document when it appears in different positions in a list. Although the order effect of relevance judgment has significant theoretical and practical implications, the extant literature is inconclusive regarding its existence and forming mechanisms. This study proposes a set of order effect forming mechanisms, including the learning effect, the subneed scheduling effect, and the cursoriness effect based on the conceptualization of dynamic relevance and the psychology of cognitive elaboration. Our empirical study indicates that in an interactive information retrieval setting, when a document list is reasonably long, order effects demonstrate a curvilinear pattern that conforms to the combined effect of the three mechanisms. Moreover, the curvilinear pattern of order effect could differ for documents of different relevance levels. © 2008 Wiley Periodicals, Inc.