Algebraic geometry and stochastic complexity of hidden Markov models

Authors:
Keisuke Yamazaki;Sumio Watanabe
Affiliations:
Precision and Intelligence Laboratory, Tokyo Institute of Technology, 4259 Nagatsuda, Midori-ku, Yokohama 226-8503, Japan;Precision and Intelligence Laboratory, Tokyo Institute of Technology, 4259 Nagatsuda, Midori-ku, Yokohama 226-8503, Japan
Venue:
Neurocomputing
Year:
2005

Citing 7
Cited 6

Bayesian interpolation

Neural Computation
Algebraic geometrical methods for hierarchical learning machines

Neural Networks
On the problem in model selection of neural network regression in overrealizable scenario

Neural Computation
Singularities in mixture models and upper bounds of stochastic complexity

Neural Networks
Algebraic Analysis for Nonidentifiable Learning Machines

Neural Computation
Asymptotic model selection for naive Bayesian networks

UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
A decision-theoretic extension of stochastic complexity and its applications to learning

IEEE Transactions on Information Theory

Equations of states in singular statistical estimation

Neural Networks
Asymptotic analysis of Bayesian generalization error with Newton diagram

Neural Networks
Experimental study of ergodic learning curve in hidden Markov models

ICONIP'08 Proceedings of the 15th international conference on Advances in neuro-information processing - Volume Part I
Extracting state transition dynamics from multiple spike trains using hidden markov models with correlated poisson distribution

Neural Computation
Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory

The Journal of Machine Learning Research
On equivalent parameter learning in simplified feature space based on Bayesian asymptotic analysis

Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

Hidden Markov models are now used in many fields, for example, speech recognition, natural language processing, etc. However, the mathematical foundation of analysis for the models is not yet constructed, since the HMM is non-identifiable. In recent years, we have developed the algebraic geometrical method that allows us to analyze the non-regular and non-identifiable models. In this paper, we apply this method to the HMM and reveal the asymptotic stochastic complexity in a mathematically rigorous way. Our results show that the Bayesian estimation makes the generalization error small and that the well known BIC is different from the stochastic complexity.