Automated evaluation of search engine performance via implicit user feedback

Authors:
Himanshu Sharma;Bernard J. Jansen
Affiliations:
The Pennsylvania State University, University Park, PA;The Pennsylvania State University, University Park, PA
Venue:
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Year:
2005

Citing 2
Cited 3

Implicit feedback for inferring user preference: a bibliography

ACM SIGIR Forum
SIGIR 2003 workshop report: implicit measures of user interests and preferences

ACM SIGIR Forum

Analysis of internet users' interests based on windows GUI messages

HCI'07 Proceedings of the 12th international conference on Human-computer interaction: applications and services
An overview of Web search evaluation methods

Computers and Electrical Engineering
Interpreting user inactivity on search results

ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

Measuring the information retrieval effectiveness of Web search engines can be expensive if human relevance judgments are required to evaluate search results. Using implicit user feedback for search engine evaluation provides a cost and time effective manner of addressing this problem. Web search engines can use human evaluation of search results without the expense of human evaluators. An additional advantage of this approach is the availability of real time data regarding system performance. Wecapture user relevance judgments actions such as print, save and bookmark, sending these actions and the corresponding document identifiers to a central server via a client application. We use this implicit feedback to calculate performance metrics, such as precision. We can calculate an overall system performance metric based on a collection of weighted metrics.