Comparing two markov methods for part-of-speech tagging of portuguese

Authors:
Fábio N. Kepler;Marcelo Finger
Affiliations:
Institute of Mathematics and Statistics, University of São Paulo (USP);Institute of Mathematics and Statistics, University of São Paulo (USP)
Venue:
IBERAMIA-SBIA'06 Proceedings of the 2nd international joint conference, and Proceedings of the 10th Ibero-American Conference on AI 18th Brazilian conference on Advances in Artificial Intelligence
Year:
2006

Citing 5
Cited 3

Grammatical category disambiguation by statistical optimization

Computational Linguistics
Building a large annotated corpus of English: the penn treebank

Computational Linguistics - Special issue on using large corpora: II
TnT: a statistical part-of-speech tagger

ANLC '00 Proceedings of the sixth conference on Applied natural language processing
A stochastic parts program and noun phrase parser for unrestricted text

ANLC '88 Proceedings of the second conference on Applied natural language processing
Feature-rich part-of-speech tagging with a cyclic dependency network

NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1

Portuguese Part-of-Speech Tagging Using Entropy Guided Transformation Learning

PROPOR '08 Proceedings of the 8th international conference on Computational Processing of the Portuguese Language
Variable-length Markov models and ambiguous words in Portuguese

YIWCALA '10 Proceedings of the NAACL HLT 2010 Young Investigators Workshop on Computational Approaches to Languages of the Americas
Variable-length Markov models and ambiguous words in Portuguese

YIWCALA '10 Proceedings of the NAACL HLT 2010 Young Investigators Workshop on Computational Approaches to Languages of the Americas

Quantified Score

Hi-index	0.00

Visualization

Abstract

There is a wide variety of statistical methods applied to Part-of-Speech (PoS) tagging, that associate words in a text to their corresponding PoS. The majority of those methods analyse a fixed, small neighborhood of words imposing some form of Markov restriction. In this work we implement and compare a fixed length hidden Markov model (HMM) with a variable length Markov chain (VLMC); the latter is, in principle, capable of detecting long distance dependencies. We show that the VLMC model performs better in terms of accuracy and almost equally in terms of tagging time, also doing very well in training time. However, the VLMC method actually fails to capture really long distance dependencies, and we analyse the reasons for such behaviour.