Is it worth submitting this run?: assess your RTE system with a good sparring partner

  • Authors:
  • Milen Kouylekov;Yashar Mehdad;Matteo Negri

  • Affiliations:
  • CELI s.r.l., Turin, Italy;FBK-irst and University of Trento, Trento, Italy;FBK-irst, Trento, Italy

  • Venue:
  • TIWTE '11 Proceedings of the TextInfer 2011 Workshop on Textual Entailment
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

We address two issues related to the development of systems for Recognizing Textual Entailment. The first is the impossibility to capitalize on lessons learned over the different datasets available, due to the changing nature of traditional RTE evaluation settings. The second is the lack of simple ways to assess the results achieved by our system on a given training corpus, and figure out its real potential on unseen test data. Our contribution is the extension of an open-source RTE package with an automatic way to explore the large search space of possible configurations, in order to select the most promising one over a given dataset. From the developers' point of view, the efficiency and ease of use of the system, together with the good results achieved on all previous RTE datasets, represent a useful support, providing an immediate term of comparison to position the results of their approach.