Using regression for spectral estimation of HMMs

Authors:
Jordan Rodu;Dean P. Foster;Weichen Wu;Lyle H. Ungar
Affiliations:
The Wharton School, Department of Statistics, University of Pennsylvania, Philadelphia, PA;The Wharton School, Department of Statistics, University of Pennsylvania, Philadelphia, PA;Computer and Information Science, University of Pennsylvania, Philadelphia, PA;Computer and Information Science, University of Pennsylvania, Philadelphia, PA
Venue:
SLSP'13 Proceedings of the First international conference on Statistical Language and Speech Processing
Year:
2013

Citing 9
Cited 0

Class-based n-gram models of natural language

Computational Linguistics
Observable Operator Models for Discrete Stochastic Time Series

Neural Computation
Summarizing speech without text using hidden Markov models

NAACL-Short '06 Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers
Hidden Markov tree model in dependency-based machine translation

ACLShort '09 Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
Open-domain semantic role labeling by modeling word spans

ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
A fast fertility hidden Markov model for word alignment using MCMC

EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
A pronoun anaphora resolution system based on factorial hidden Markov models

HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Spectral learning for non-deterministic dependency parsing

EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
Spectral learning of latent-variable PCFGs

ACL '12 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1

Quantified Score

Hi-index	0.00

Visualization

Abstract

Hidden Markov Models (HMMs) are widely used to model discrete time series data, but the EM and Gibbs sampling methods used to estimate them are often slow or prone to get stuck in local minima. A more recent class of reduced-dimension spectral methods for estimating HMMs has attractive theoretical properties, but their finite sample size behavior has not been well characterized. We introduce a new spectral model for HMM estimation, a corresponding spectral bilinear regression model, and systematically compare them with a variety of competing simplified models, explaining when and why each method gives superior performance. Using regression to estimate HMMs has a number of advantages, allowing more powerful and flexible modeling.