Designing search engine retrieval effectiveness tests with RAT

Authors:
Dirk Lewandowski;Sebastian Sünkler
Affiliations:
Department of Information, Hamburg University of Applied Sciences, Hamburg, Germany. E-mails: {dirk.lewandowski, sebastian.suenkler}@haw-hamburg.de;Department of Information, Hamburg University of Applied Sciences, Hamburg, Germany. E-mails: {dirk.lewandowski, sebastian.suenkler}@haw-hamburg.de
Venue:
Information Services and Use - APE 2013 --The Funding of Publishing
Year:
2013

Citing 7
Cited 0

The pragmatics of information retrieval experimentation, revisited

Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
Finding information on the World Wide Web: the retrieval effectiveness of search engines

Information Processing and Management: an International Journal
Measuring Search Engine Quality

Information Retrieval
The effectiveness of web search engines for retrieving relevant ecommerce links

Information Processing and Management: an International Journal
What users see - Structures in search engine results pages

Information Sciences: an International Journal
The influence of commercial intent of search results on their perceived relevance

Proceedings of the 2011 iConference
Search-logger analyzing exploratory search tasks

Proceedings of the 2011 ACM Symposium on Applied Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We introduce the Relevance Assessment Tool RAT, which allows researchers to design complex search engine retrieval effectiveness studies. This Web-based tool consists of different modules that researchers can use to design tests, to collect results from different search engines using a screen-scraping approach, and to collect judgments from a multitude of jurors using a crowdsourcing approach. We designed the software as a Web-based application, which allows for a distributed collection of relevance judgments, and the like. Using the Relevance Assessment Tool allows for much larger search engine studies than previously conducted. To our knowledge, there is no comparable tool that was developed in the academic context. Therefore, no information regarding the design of such software was available to-date.