Some thoughts on the reported results of TREC

Authors:
David C. Blair
Affiliations:
Computer and Information Systems, Graduate School of Business, University of Michigan, Ann Arbor, MI
Venue:
Information Processing and Management: an International Journal
Year:
2002

Citing 12
Cited 5

Language and representation in information retrieval

Language and representation in information retrieval
SILOL: a simple logical-linguistic document retrieval system

Information Processing and Management: an International Journal - Special issue on natural language processing and information retrieval
Artificial intelligence (3rd ed.)

Artificial intelligence (3rd ed.)
STAIRS redux: thoughts on the STAIRS evaluation, ten years after

Journal of the American Society for Information Science - Special issue: evaluation of information retrieval systems
How reliable are the results of large-scale information retrieval experiments?

Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Overview of the sixth text REtrieval conference (TREC-6)

Information Processing and Management: an International Journal - The sixth text REtrieval conference (TREC-6)
Further reflections on TREC

Information Processing and Management: an International Journal - The sixth text REtrieval conference (TREC-6)
Experimentation as a way of life: Okapi at TREC

Information Processing and Management: an International Journal - The sixth text REtrieval conference (TREC-6)
Corporate Intranet

Corporate Intranet
The challenge of commercial document retrieval, part I: major issues, and a framework based on search exhaustivity, determinacy of representation and document collection size

Information Processing and Management: an International Journal
The challenge of commercial document retrieval, part II: a strategy for document searching based on identifiable document partitions

Information Processing and Management: an International Journal
The Text REtrieval Conferences (TRECs)

TIPSTER '96 Proceedings of a workshop on held at Vienna, Virginia: May 6-8, 1996

The challenge of commercial document retrieval, part I: major issues, and a framework based on search exhaustivity, determinacy of representation and document collection size

Information Processing and Management: an International Journal
New measurements for search engine evaluation proposed and tested

Information Processing and Management: an International Journal
Evaluating epistemic uncertainty under incomplete assessments

Information Processing and Management: an International Journal
On the history of evaluation in IR

Journal of Information Science
A retrieval evaluation methodology for incomplete relevance assessments

ECIR'07 Proceedings of the 29th European conference on IR research

Quantified Score

Hi-index	0.01

Visualization

Abstract

The periodic TRECs - Text REtrieval Conferences - have reported the results of a variety of recall studies in large-scale document retrieval. While the efforts of TREC are noteworthy and laudable, there are reasons why its results, especially the recall values which are central to its conclusions, should be accepted with some caution.