Panel: building and using test collections
SIGIR '96 Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval
| Hi-index | 0.00 |
Recently we have been investigating various methods for improving the retrieval of documents. One frequently used context for such studies is to work with a known database and set of queries, where sufficient work has previously been done to establish an "official" set of relevant documents for each query. Working within such a context, we have discovered at least one instance wherein the combination of the document descriptions in the database, the query, and the set of documents deemed relevant foredooms our attempts at retrieval improvement to failure. This note is to caution other researchers against such possibilities, and to recommend that the database in question, that of the National Physical Laboratory (NPL), as represented on Virginia Disk 1, be retired from use.