Mining natural language answers from the web

  • Authors:
  • Günter Neumann;Feiyu Xu

  • Affiliations:
  • Language Technology Lab, DFKI, D-66123 Saarbrücken, Germany;Language Technology Lab, DFKI, D-66123 Saarbrücken, Germany

  • Venue:
  • Web Intelligence and Agent Systems
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a novel method for mining textual answers in Web pages using semi-structured NL questions and Google for initial document retrieval. We exploit the redundancy on the Web by weighting all identified named entities (NEs) found in the relevant document set based on their occurrences and distributions. The ranked NEs are used as our primary anchors for document indexing, paragraph selection, and answer identification. The latter is dependent on two factors: the overlap of terms at different levels (e.g., tokens and named entities) between queries and sentences, and the relevance of identified NEs corresponding to the expected answer type. The set of answer candidates is further subdivided into ranked equivalent classes from which the final answer is selected. The system has been evaluated using question-answer pairs extracted from a popular German quiz book.