Creating test collections from user generated content for GIR evaluation

Authors:
Damien Palacio;Curdin Derungs;Ross Purves
Affiliations:
University of Zurich, Switzerland;University of Zurich, Switzerland;University of Zurich, Switzerland
Venue:
Proceedings of the 7th Workshop on Geographic Information Retrieval
Year:
2013

Citing 3
Cited 0

Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?

Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Information retrieval system evaluation: effort, sensitivity, and reliability

Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
GeoCLEF 2008: the CLEF 2008 cross-language geographic information retrieval track overview

CLEF'08 Proceedings of the 9th Cross-language evaluation forum conference on Evaluating systems for multilingual and multimodal information access

Quantified Score

Hi-index	0.00

Visualization

Abstract

Evaluation of the effectiveness of Geographic Information Retrieval (GIR) systems is challenging and time consuming. We describe an approach to such evaluations, where we use user generated content in the form of text and associated metadata to build a large test colletion automatically. We can thus show that the UGC test collection is useful for evaluating and exploring some of the critical aspects of a GIR, for instance by submitting large numbers of queries.