The effect of topic set size on retrieval experiment error
SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Exploring the computing literature with visualization and stepping stones & pathways
Communications of the ACM - Supporting exploratory search
Find-similar: similarity browsing as a search tool
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Speeding Coordination by Combining Analytical and Inductive Learning
WI-IATW '07 Proceedings of the 2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops
How robust are multilingual information retrieval systems?
Proceedings of the 2008 ACM symposium on Applied computing
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Query Expansion Using External Evidence
ECIR '09 Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval
System scoring using partial prior information
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Query hardness estimation using Jensen-Shannon divergence among multiple scoring functions
ECIR'07 Proceedings of the 29th European conference on IR research
Evaluating text representations for retrieval of the best group of documents
ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
Using coherence-based measures to predict query difficulty
ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
CLEF 2008: ad hoc track overview
CLEF'08 Proceedings of the 9th Cross-language evaluation forum conference on Evaluating systems for multilingual and multimodal information access
PRES: a score metric for evaluating recall-oriented information retrieval applications
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
A topological embedding of the lexicon for semantic distance computation
Natural Language Engineering
CLEF 2009 ad hoc track overview: robust-WSD task
CLEF'09 Proceedings of the 10th cross-language evaluation forum conference on Multilingual information access evaluation: text retrieval experiments
Improving effectiveness of query expansion using information theoretic approach
IEA/AIE'10 Proceedings of the 23rd international conference on Industrial engineering and other applications of applied intelligent systems - Volume Part II
CLEF'05 Proceedings of the 6th international conference on Cross-Language Evalution Forum: accessing Multilingual Information Repositories
On effectiveness measures and relevance functions in ranking INEX systems
AIRS'05 Proceedings of the Second Asia conference on Asia Information Retrieval Technology
Updating broken web links: An automatic recommendation system
Information Processing and Management: an International Journal
Measuring the variability in effectiveness of a retrieval system
IRFC'10 Proceedings of the First international Information Retrieval Facility conference on Adbances in Multidisciplinary Retrieval
An information retrieval approach based on discourse type
NLDB'06 Proceedings of the 11th international conference on Applications of Natural Language to Information Systems
Robust ranking models via risk-sensitive optimization
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Automatic query performance assessment during the retrieval of software artifacts
Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering
CLEF 2006: ad hoc track overview
CLEF'06 Proceedings of the 7th international conference on Cross-Language Evaluation Forum: evaluation of multilingual and multi-modal information retrieval
Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks
CLEF'06 Proceedings of the 7th international conference on Cross-Language Evaluation Forum: evaluation of multilingual and multi-modal information retrieval
Robust query rewriting using anchor data
Proceedings of the sixth ACM international conference on Web search and data mining
An argumentation-oriented multi-agent system for automating the freight planning process
Expert Systems with Applications: An International Journal
Effective and Robust Query-Based Stemming
ACM Transactions on Information Systems (TOIS)
| Hi-index | 0.00 |
The robust retrieval track explores methods for improving the consistency of retrieval technology by focusing on poorly performing topics. The retrieval task in the track is a traditional ad hoc retrieval task where the evaluation methodology emphasizes a system's least effective topics. The most promising approach to improving poorly performing topics is exploiting text collections other than the target collection such as the web.The track has also investigated appropriate evaluation measures to support the focus on ineffective topics. Traditional measure are dominated by the better-performing topics, and the first two measures used in the track that do emphasize the poorly performing topics are unstable in practice. A third measure, a variant of the traditional MAP measure that uses a geometric mean rather than an arithmetic mean to average individual topic results, shows promise of giving appropriate emphasis to poorly performing topics while being more stable at equal topic set sizes.