Pattern Recognition for Conditionally Independent Data
The Journal of Machine Learning Research
Another look at indirect negative evidence
CACLA '09 Proceedings of the EACL 2009 Workshop on Cognitive Aspects of Computational Language Acquisition
Generalization performance of ERM algorithm with geometrically ergodic Markov chain samples
ICNC'09 Proceedings of the 5th international conference on Natural computation
Generalization bounds of ERM algorithm with V-geometrically Ergodic Markov chains
Advances in Computational Mathematics
| Hi-index | 754.84 |
We consider a model of learning in which the successive observations follow a certain Markov chain. The observations are labeled according to a membership to some unknown target set. For a Markov chain with finitely many states we show that, if the target set belongs to a family of sets with a finite Vapnik-Chervonenkis (1995) dimension, then probably approximately correct (PAC) learning of this set is possible with polynomially large samples. Specifically for observations following a random walk with a state space 𝒳 and uniform stationary distribution, the sample size required is no more than Ω(t0/1-λ2log(t0|χ|1/δ)), where δ is the confidence level, λ2 is the second largest eigenvalue of the transition matrix, and t0 is the sample size sufficient for learning from independent and identically distributed (i.i.d.) observations. We then obtain similar results for Markov chains with countably many states using Lyapunov function technique and results on mixing properties of infinite state Markov chains.