Industrial evaluation of a highly-accurate academic IR system

  • Authors:
  • Tahia Infantes-Morris;Philip J. Bernhard;Kevin L. Fox;Gary J. Faulkner;Kristina Stripling

  • Affiliations:
  • Harris Government Communications Systems Division;Harris Government Communications Systems Division;Harris Government Communications Systems Division;Harris Government Communications Systems Division;Harris Government Communications Systems Division

  • Venue:
  • CIKM '03 Proceedings of the twelfth international conference on Information and knowledge management
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper we report the results of an independent experimental evaluation of an information retrieval (IR) system developed at the Illinois Institute of Technology (IIT). The system, which is called the Advanced Information Retrieval Engine (AIRE), consists of a set of tools and utilities providing indexing, extraction, searching and visualization. We evaluated AIRE on three data sets from the Text REtrieval Conference (TREC) - TREC 8, 9 and 10. Overall, our results indicate that AIRE is a highly accurate IR system. Compared with results published by IIT, in our experiments AIRE consistently scored higher in recall. AIRE also scored higher in precision, but only for automatic tasks. In manual tasks, AIRE scored lower in precision in our experiments, but we attributed that to factors external to AIRE. Our final conclusion is that AIRE is a highly accurate IR system.