A benchmark evaluation of the french MeSH indexers

Authors:
Aurélie Névéol;Vincent Mary;Arnaud Gaudinat;Célia Boyer;Alexandrina Rogozan;Stéfan J. Darmoni
Affiliations:
PSI Laboratory, Rouen, France;Rennes Medical School, France;HON Foundation, Geneva, Switzerland;HON Foundation, Geneva, Switzerland;PSI Laboratory, Rouen, France;PSI Laboratory, Rouen, France
Venue:
AIME'05 Proceedings of the 10th conference on Artificial Intelligence in Medicine
Year:
2005

Citing 2
Cited 1

Knowledge-based indexing of the medical literature: the indexing aid project

Journal of the American Society for Information Science
Automatic indexing of online health resources for a French quality controlled gateway

Information Processing and Management: an International Journal

An approach based on langage modeling for improving biomedical information retrieval

International Journal of Knowledge-based and Intelligent Engineering Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

The increasing demand on both practitioners and librarians to encode medical documents with controlled vocabularies calls for automatic tools and methods to help them perform this task efficiently. This paper presents the Benchmark evaluation of the French MeSH indexing systems carried out under the umbrella of the VUMeF consortium. The CISMeF, NOMINDEX and HONMeSHMapper systems are introduced, and evaluated on a set of 82 resources randomly taken from the CISMeF catalogue. The automatic MeSH indexing produced by each system was compared to the manual gold standard provided by the CISMeF medical librarian team. The automatic systems achieve at best a precision close to 50% at rank 1 (HONMeSHMapper, CISMeF) and HONMeSHMapper achieves the best overall F-measure. A qualitative evaluation of the indexing provided indicates that all systems tend to misevaluate the specificity of the terms to retrieve.