Automatic lemmatizer construction with focus on OOV words lemmatization

Authors:
Jakub Kanis;Luděk Müller
Affiliations:
Department of Cybernetics, University of West Bohemia, Plzeň, Czech Republic;Department of Cybernetics, University of West Bohemia, Plzeň, Czech Republic
Venue:
TSD'05 Proceedings of the 8th international conference on Text, Speech and Dialogue
Year:
2005

Citing 1
Cited 2

Viewing morphology as an inference process

SIGIR '93 Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval

Automatic training of lemmatization rules that handle morphological changes in pre-, in- and suffixes alike

ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - Volume 1
Comparison of different lemmatization approaches through the means of information retrieval performance

TSD'10 Proceedings of the 13th international conference on Text, speech and dialogue

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper deals with the automatic construction of a lemmatizer from a Full Form – Lemma (FFL) training dictionary and with lemmatization of new, in the FFL dictionary unseen, i.e. out-of-vocabulary (OOV) words. Three methods of lemmatization of three kinds of OOV words (missing full forms, unknown words, and compound words) are introduced. These methods were tested on Czech test data. The best result (recall: 99.3 % and precision: 75.1 %) has been achieved by a combination of these methods. The lexicon-free lemmatizer based on the method of lemmatization of unknown words (lemmatization patterns method) is introduced too.