Open- vs. Restricted-Domain QA Systems in the Biomedical Field

Authors:
María-Dolores Olvera-Lobo;Juncal Gutiérrez-Artacho
Affiliations:
CSIC, Unidad Asociada Grupo SCImago, Madrid, Spain andUniversidad de Granada, Department of Library and Information Science, Granada,Spain;Universidad de Granada, Department of Translation andInterpretation, Granada, Spain
Venue:
Journal of Information Science
Year:
2011

Citing 8
Cited 2

Evaluation of evaluation in information retrieval

SIGIR '95 Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval
Information storage and retrieval

Information storage and retrieval
Modern Information Retrieval

Modern Information Retrieval
Omnibase: Uniform Access to Heterogeneous Data for Question Answering

NLDB '02 Proceedings of the 6th International Conference on Applications of Natural Language to Information Systems-Revised Papers
New Directions in Question Answering

New Directions in Question Answering
Development, implementation, and a cognitive evaluation of a definitional question answering system for physicians

Journal of Biomedical Informatics
Ontology-based information extraction and integration from heterogeneous data sources

International Journal of Human-Computer Studies
The QuALiM question answering demo: supplementing answers with paragraphs drawn from Wikipedia

HLT-Demonstrations '08 Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Demo Session

Health conversational system based on contextual matching of community-driven question-answer pairs

Proceedings of the 20th ACM international conference on Information and knowledge management
Contextual question answering for the health domain

Journal of the American Society for Information Science and Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

Question answering systems (QA systems) stand as a new alternative for information retrieval systems. We conducted a study to evaluate the efficiency of QA systems as terminological sources for physicians, specialized translators and users in general. To this end we analysed the performance of two open-domain and two restricted-domain QA systems. The research entailed a collection of 150 definitional questions from WebMed. We studied the sources that QA systems used to retrieve the answers, and later applied a range of evaluation measures to mark the quality of answers. Through analysing the results obtained by asking the 150 questions in the QA systems MedQA, START, QuALiM and HONqa, it was possible to evaluate the systems芒聙聶 operation through applying specific metrics (MRR, FHS, TRR, Precision, Recall). Despite the limitations demonstrated by these systems, it has been confirmed that these four QA systems are valid and useful for obtaining definitional medical information in that they offer coherent and precise answers.