Generating an entailment corpus from news headlines

  • Authors:
  • John Burger;Lisa Ferro

  • Affiliations:
  • The MITRE Corporation, Bedford, MA;The MITRE Corporation, Bedford, MA

  • Venue:
  • EMSEE '05 Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

We describe our efforts to generate a large (100,000 instance) corpus of textual entailment pairs from the lead paragraph and headline of news articles. We manually inspected a small set of news stories in order to locate the most productive source of entailments, then built an annotation interface for rapid manual evaluation of further exemplars. With this training data we built an SVM-based document classifier, which we used for corpus refinement purposes---we believe that roughly three-quarters of the resulting corpus are genuine entailment pairs. We also discuss the difficulties inherent in manual entailment judgment, and suggest ways to ameliorate some of these.