The effect of topic set size on retrieval experiment error
SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Dynamic test collections: measuring search effectiveness on the live web
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
On the reliability of factoid question answering evaluation
ACM Transactions on Asian Language Information Processing (TALIP)
Exploiting underrepresented query aspects for automatic query expansion
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Information Processing and Management: an International Journal
Multinomial randomness models for retrieval with document fields
ECIR'07 Proceedings of the 29th European conference on IR research
Combining evidence for relevance criteria: a framework and experiments in web retrieval
ECIR'07 Proceedings of the 29th European conference on IR research
An overview of Web search evaluation methods
Computers and Electrical Engineering
On effectiveness measures and relevance functions in ranking INEX systems
AIRS'05 Proceedings of the Second Asia conference on Asia Information Retrieval Technology
Bootstrap-Based comparisons of IR metrics for finding one relevant document
AIRS'06 Proceedings of the Third Asia conference on Information Retrieval Technology
| Hi-index | 0.00 |
Many common web searches by their nature have a very small number of relevant documents. Homepage and "namedpage" searching are known-item searches where there is only a single relevant document. Topic distillation is a special kind of topical relevance search where the user wishes to find a few key web sites rather than every relevant web page. Because these types of searches are so common, web search evaluations have come to focus on tasks where there are very few relevant documents. Evaluations with few relevant documents pose special challenges for current metrics. In particular, the TREC 2003 topic distillation evaluation is unable to distinguish most submitted runs from each other.