The text REtrieval conference (TREC): history and plans for TREC-9

  • Authors:
  • Ellen M. Voorhees;Donna Harman

  • Affiliations:
  • National Institute of Standards and Technology, 100 Bureau Drive, STOP 8940, Gaithersburg, MD;National Institute of Standards and Technology, 100 Bureau Drive, STOP 8940, Gaithersburg, MD

  • Venue:
  • ACM SIGIR Forum
  • Year:
  • 1999

Quantified Score

Hi-index 0.00

Visualization

Abstract

Text retrieval systems have historically been refined through experimentation on test collections. In 1990 NIST was asked to build a very large test collection for use in evaluation of text retrieval technology in the DARPA TIPSTER project. This collection was to be on the order of 1 million full-text documents: a magnitude about 100 times larger than any existing non-proprietary test collection. The following year NIST proposed that this collection be made available to the full research community by the formation of the Text REtrieval Conference (TREC). The first conference took place in September, 1992 with 25 participating groups including most of the leading text retrieval research groups. Although scaling their research algorithms to handle this near-operational amount of text (for 1992) was a Herculean task, groups joined TREC to get the test data and to take part in the first major cross-system search engine evaluation. Participation in TREC has since increased each year. The most recent TREC (TREC-8 held in November, 1999) had 66 participating groups representing 16 different countries.This paper serves as a general introduction to TREC. Detailed information about TREC including the complete Proceedings for each workshop, instructions for obtaining test collections, and the Call for Participation for TREC-9 can be found on the TREC web site at http://trec.nist.gov. Task descriptions for some of the tasks to be done in TREC-9 follow in this issue. To participate in TREC-9, follow the instructions given in the Call for Participation.