The study of a nonstationary maximum entropy Markov model and its application on the pos-tagging task

Authors:
Jinghui Xiao;Xiaolong Wang;Bingquan Liu
Affiliations:
Harbin Institute of Technology, Harbin, China, P.C.;Harbin Institute of Technology, Harbin, China, P.C.;Harbin Institute of Technology, Harbin, China, P.C.
Venue:
ACM Transactions on Asian Language Information Processing (TALIP)
Year:
2007

Citing 16
Cited 0

Elements of information theory

Elements of information theory
Class-based n-gram models of natural language

Computational Linguistics
Nonstationary hidden Markov model

Signal Processing
Inducing Features of Random Fields

IEEE Transactions on Pattern Analysis and Machine Intelligence
An Omnifont Open-Vocabulary OCR System for English and Arabic

IEEE Transactions on Pattern Analysis and Machine Intelligence
Foundations of statistical natural language processing

Foundations of statistical natural language processing
Nonstationary-state hidden Markov model representation of speech signals for speech enhancement

Signal Processing
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Maximum Entropy Markov Models for Information Extraction and Segmentation

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Tutorial on maximum likelihood estimation

Journal of Mathematical Psychology
Hidden Markov models with states depending on observations

Pattern Recognition Letters
Discriminative syntactic language modeling for speech recognition

ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Minimum sample risk methods for language modeling

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Smoothing methods in maximum entropy language modeling

ICASSP '99 Proceedings of the Acoustics, Speech, and Signal Processing, 1999. on 1999 IEEE International Conference - Volume 01
Principles of non-stationary hidden markov model and its applications to sequence labeling task

IJCNLP'05 Proceedings of the Second international joint conference on Natural Language Processing
An MCMC sampling approach to estimation of nonstationary hiddenMarkov models

IEEE Transactions on Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Sequence labeling is a core task in natural language processing. The maximum entropy Markov model (MEMM) is a powerful tool in performing this task. This article enhances the traditional MEMM by exploiting the positional information of language elements. The stationary hypothesis is relaxed in MEMM, and the nonstationary MEMM (NS-MEMM) is proposed. Several related issues are discussed in detail, including the representation of positional information, NS-MEMM implementation, smoothing techniques, and the space complexity issue. Furthermore, the asymmetric NS-MEMM presents a more flexible way to exploit positional information. In the experiments, NS-MEMM is evaluated on both the Chinese and the English pos-tagging tasks. According to the experimental results, NS-MEMM yields effective improvements over MEMM by exploiting positional information. The smoothing techniques in this article effectively solve the NS-MEMM data-sparseness problem; the asymmetric NS-MEMM is also an improvement by exploiting positional information in a more flexible way.