Two levels of evaluation in a complex NL system

Authors:
Jean-Baptiste Berthelin;Brigitte Grau;Martine Hurault-Plantet
Affiliations:
LIMSI-CNRS, Orsay, France;LIMSI-CNRS, Orsay, France;LIMSI-CNRS, Orsay, France
Venue:
ELDS '01 Proceedings of the workshop on Evaluation for Language and Dialogue Systems - Volume 9
Year:
2001

Citing 3
Cited 0

Boosting variant recognition with light semantics

COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
Experiments with open-domain textual Question Answering

COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
Syntagmatic and paradigmatic representations of term variation

ACL '99 Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics

Quantified Score

Hi-index	0.00

Visualization

Abstract

The QALC question-answering system, developed at LIMSI, has been a participant for two years in the QA track of the TREC conference. In this paper, we present a quantitative evaluation of various modules in our system, based on two criteria: first, the numbers of documents containing the correct answer and selected by the system; secondly, the number of answers found. The first criterion is used for evaluating locally the modules in the system, which contribute in selecting documents that are likely to contain the answer. The second one provides a global evaluation of the system. As such, it also serves for an indirect evaluation of various modules.