Universal Compression and Retrieval
Universal Compression and Retrieval
Information Theory and Reliable Communication
Information Theory and Reliable Communication
Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probability
Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probability
Prediction, Learning, and Games
Prediction, Learning, and Games
On the possibility of learning in reactive environments with arbitrary dependence
Theoretical Computer Science
Characterizing predictable classes of processes
UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
A general minimax result for relative entropy
IEEE Transactions on Information Theory
On the Relation between Realizable and Nonrealizable Cases of the Sequence Prediction Problem
The Journal of Machine Learning Research
Hi-index | 0.00 |
The problem is sequence prediction in the following setting. A sequence x1,...,xn,... of discrete-valued observations is generated according to some unknown probabilistic law (measure) μ. After observing each outcome, it is required to give the conditional probabilities of the next observation. The measure μ belongs to an arbitrary but known class C of stochastic process measures. We are interested in predictors ρ whose conditional probabilities converge (in some sense) to the "true" μ-conditional probabilities, if any μ∈C is chosen to generate the sequence. The contribution of this work is in characterizing the families C for which such predictors exist, and in providing a specific and simple form in which to look for a solution. We show that if any predictor works, then there exists a Bayesian predictor, whose prior is discrete, and which works too. We also find several sufficient and necessary conditions for the existence of a predictor, in terms of topological characterizations of the family C, as well as in terms of local behaviour of the measures in C, which in some cases lead to procedures for constructing such predictors. It should be emphasized that the framework is completely general: the stochastic processes considered are not required to be i.i.d., stationary, or to belong to any parametric or countable family.