Experimental study of ergodic learning curve in hidden Markov models

Authors:
Masashi Matsumoto;Sumio Watanabe
Affiliations:
Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology, Yokohama, Japan;Precision and Intelligence Laboratory, Tokyo Institute of Technology, Yokohama, Japan
Venue:
ICONIP'08 Proceedings of the 15th international conference on Advances in neuro-information processing - Volume Part I
Year:
2008

Citing 5
Cited 0

Algebraic geometrical methods for hierarchical learning machines

Neural Networks
Singularities in mixture models and upper bounds of stochastic complexity

Neural Networks
Algebraic Analysis for Nonidentifiable Learning Machines

Neural Computation
Stochastic complexities of reduced rank regression in Bayesian estimation

Neural Networks
Algebraic geometry and stochastic complexity of hidden Markov models

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

A number of learning machines used in information science are not regular, but rather singular, because they are non-identifiable and their Fisher information matrices are singular. Even for singular learning machines, the learning theory was developed for the case in which training samples are independent. However, if training samples have time-dependency, then learning theory is not yet established. In the present paper, we define an ergodic generalization error for a time-dependent sequence and study its behavior experimentally in hidden Markov models. The ergodic generalization error is clarified to be inversely proportional to the number of training samples, but the learning coefficient depends strongly on time-dependency.