Arabic diacritization using weighted finite-state transducers

Authors:
Rani Nelken;Stuart M. Shieber
Affiliations:
Harvard University, Cambridge, MA;Harvard University, Cambridge, MA
Venue:
Semitic '05 Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages
Year:
2005

Citing 7
Cited 10

A design principles of a weighted finite-state transducer library

Theoretical Computer Science - Special issue on implementing automata
Generalized algorithms for constructing statistical language models

ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
An HMM approach to vowel restoration in Arabic and Hebrew

SEMITIC '02 Proceedings of the ACL-02 workshop on Computational approaches to semitic languages
Machine transliteration of names in Arabic text

SEMITIC '02 Proceedings of the ACL-02 workshop on Computational approaches to semitic languages
Automatic tagging of Arabic text: from raw text to base phrase chunks

HLT-NAACL-Short '04 Proceedings of HLT-NAACL 2004: Short Papers
Issues in Arabic orthography and morphology analysis

Semitic '04 Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages
Automatic diacritization of Arabic for acoustic modeling in speech recognition

Semitic '04 Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages

Maximum entropy based restoration of Arabic diacritics

ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Text Entry Systems: Mobility, Accessibility, Universality

Text Entry Systems: Mobility, Accessibility, Universality
Constrained Sequence Classification for Lexical Disambiguation

PRICAI '08 Proceedings of the 10th Pacific Rim International Conference on Artificial Intelligence: Trends in Artificial Intelligence
Arabic diacritic restoration approach based on maximum entropy models

Computer Speech and Language
Arabic tokenization system

Semitic '07 Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources
Automatic diacritization for low-resource languages using a hybrid word and consonant CMM

HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Handling unknown words in statistical latent-variable parsing models for Arabic, English and French

SPMRL '10 Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages
Decision trees for lexical smoothing in statistical machine translation

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
Combination of information retrieval methods with LESK algorithm for Arabic word sense disambiguation

Artificial Intelligence Review
Part of speech tagging for arabic

Natural Language Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

Arabic is usually written without short vowels and additional diacritics, which are nevertheless important for several applications. We present a novel algorithm for restoring these symbols, using a cascade of probabilistic finite-state transducers trained on the Arabic treebank, integrating a word-based language model, a letter-based language model, and an extremely simple morphological model. This combination of probabilistic methods and simple linguistic information yields high levels of accuracy.