Quantitative evaluation of passage retrieval algorithms for question answering

  • Authors:

  • Stefanie Tellex;Boris Katz;Jimmy Lin;Aaron Fernandes;Gregory Marton

  • Affiliations:

  • MIT Artificial Intelligence Laboratory, Cambridge, MA;MIT Artificial Intelligence Laboratory, Cambridge, MA;MIT Artificial Intelligence Laboratory, Cambridge, MA;MIT Artificial Intelligence Laboratory, Cambridge, MA;MIT Artificial Intelligence Laboratory, Cambridge, MA

  • Venue:
  • Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

Passage retrieval is an important component common to many question answering systems. Because most evaluations of question answering systems focus on end-to-end performance, comparison of common components becomes difficult. To address this shortcoming, we present a quantitative evaluation of various passage retrieval algorithms for question answering, implemented in a framework called Pauchok. We present three important findings: Boolean querying schemes perform well in the question answering task. The performance differences between various passage retrieval algorithms vary with the choice of document retriever, which suggests significant interactions between document retrieval and passage retrieval. The best algorithms in our evaluation employ density-based measures for scoring query terms. Our results reveal future directions for passage retrieval and question answering.