A comparative investigation of morphological language modeling for the languages of the European union

Authors:
Thomas Müller;Hinrich Schütze;Helmut Schmid
Affiliations:
University of Stuttgart, Germany;University of Stuttgart, Germany;University of Stuttgart, Germany
Venue:
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Year:
2012

Citing 9
Cited 0

Dimensions of meaning

Proceedings of the 1992 ACM/IEEE conference on Supercomputing
Class-based n-gram models of natural language

Computational Linguistics
A maximum entropy approach to natural language processing

Computational Linguistics
Factored language models and generalized parallel backoff

NAACL-Short '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume of the Proceedings of HLT-NAACL 2003--short papers - Volume 2
Unsupervised models for morpheme segmentation and morphology learning

ACM Transactions on Speech and Language Processing (TSLP)
Morph-based speech recognition and modeling of out-of-vocabulary words across languages

ACM Transactions on Speech and Language Processing (TSLP)
Modeling morphologically rich languages using split words and unstructured dependencies

ACLShort '09 Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
From frequency to meaning: vector space models of semantics

Journal of Artificial Intelligence Research
Half-context language models

Computational Linguistics

Quantified Score

Hi-index	0.00

Visualization

Abstract

We investigate a language model that combines morphological and shape features with a Kneser-Ney model and test it in a large crosslingual study of European languages. Even though the model is generic and we use the same architecture and features for all languages, the model achieves reductions in perplexity for all 21 languages represented in the Europarl corpus, ranging from 3% to 11%. We show that almost all of this perplexity reduction can be achieved by identifying suffixes by frequency.