An evaluation of retrieval effectiveness for a full-text document-retrieval system
Communications of the ACM
Another look at automatic text-retrieval systems
Communications of the ACM
Annual review of information science and technology, vol. 22
Language and representation in information retrieval
Language and representation in information retrieval
Full-text information retrieval: further analysis and clarification
Information Processing and Management: an International Journal
Knapsack problems: algorithms and computer implementations
Knapsack problems: algorithms and computer implementations
The state of retrieval system evaluation
Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
STAIRS redux: thoughts on the STAIRS evaluation, ten years after
Journal of the American Society for Information Science - Special issue: evaluation of information retrieval systems
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
The impact of query structure and query expansion on retrieval performance
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Text retrieval and filtering: analytic models of performance
Text retrieval and filtering: analytic models of performance
A novel method for the evaluation of Boolean query effectiveness across a wide operational range
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Online Information Retrieval: Concepts, Principles, and Techniques
Online Information Retrieval: Concepts, Principles, and Techniques
The Text REtrieval Conferences (TRECs)
TIPSTER '96 Proceedings of a workshop on held at Vienna, Virginia: May 6-8, 1996
Cumulated gain-based evaluation of IR techniques
ACM Transactions on Information Systems (TOIS)
Proceedings of the 24th BCS-IRSG European Colloquium on IR Research: Advances in Information Retrieval
Bibliographic database access using free-text and controlled vocabulary: an evaluation
Information Processing and Management: an International Journal
| Hi-index | 0.01 |
The STAIRS study conducted by Blair and Maron in the mid-80's is a milestone in the history of IR evaluation. Blair and Maron made strong conclusion about the inadequacy of free-text searching large databases, and their study has been widely referred in the literature to justify the problems of effectiveness in IR systems. However, some critics of the study have plausibly pointed out that the ineffectiveness conclusions were not solidly based on empirical data.This paper introduces a new theoretical and empirical approach to study the problems of high recall searching in large databases and reports the results of a case experiment. The findings verify some of the hypothetical conclusions introduced in the STAIRS study, and expands the picture of falling performance. It is shown that low precision in high recall searching is unavoidable in exact-match Boolean searching since even major concepts are often expressed implicitly in relevant documents. The author suggests that the problem could be reduced in facet-based best-match searching.