Question answering system for incomplete and noisy data: methods and measures for its evaluation

Authors:
Lili Aunimo;Oskari Heinonen;Reeta Kuuskoski;Juha Makkonen;Renaud Petit;Otso Virtanen
Affiliations:
University of Helsinki, Department of Computer Science, University of Helsinki, Finland;University of Helsinki, Department of Computer Science, University of Helsinki, Finland;University of Helsinki, Department of Computer Science, University of Helsinki, Finland;University of Helsinki, Department of Computer Science, University of Helsinki, Finland;University of Helsinki, Department of Computer Science, University of Helsinki, Finland;University of Helsinki, Department of Computer Science, University of Helsinki, Finland
Venue:
ECIR'03 Proceedings of the 25th European conference on IR research
Year:
2003

Citing 6
Cited 1

Automatic text processing: the transformation, analysis, and retrieval of information by computer

Automatic text processing: the transformation, analysis, and retrieval of information by computer
Empirical methods for artificial intelligence

Empirical methods for artificial intelligence
Assessing agreement on classification tasks: the kappa statistic

Computational Linguistics
From Plain Character Strings to Meaningful Words: Producing Better Full Text Databases for Inflectional and Compounding Languages with Morphological Analysis Software

Information Retrieval
Message classification in the call center

ANLC '00 Proceedings of the sixth conference on Applied natural language processing
A non-projective dependency parser

ANLC '97 Proceedings of the fifth conference on Applied natural language processing

An approach for adding noise-tolerance to restricted-domain information retrieval

NLDB'10 Proceedings of the Natural language processing and information systems, and 15th international conference on Applications of natural language to information systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a question answering system that can handle noisy and incomplete natural language data, and methods and measures for the evaluation of question answering systems. Our question answering system is based on the vector space model and linguistic analysis of the natural language data. In the evaluation procedure, we test eight different preprocessing schemes for the data, and come to the conclusion that lemmatization combined with breaking compound words into their constituents gives significantly better results than the baseline. The evaluation process is based on stratified random sampling and bootstrapping. To measure the correctness of an answer, we use partial credits as well as full credits.