Online learning of conditionally I.I.D. data
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Experimental investigation of forecasting methods based on data compression algorithms
Problems of Information Transmission
Pattern Recognition for Conditionally Independent Data
The Journal of Machine Learning Research
IEEE Transactions on Information Theory
Intermittent estimation for Gaussian processes
IEEE Transactions on Information Theory
Long-term prediction intervals of time series
IEEE Transactions on Information Theory
| Hi-index | 755.02 |
The conditional distribution of the next outcome given the infinite past of a stationary process can be inferred from finite but growing segments of the past. Several schemes are known for constructing pointwise consistent estimates, but they all demand prohibitive amounts of input data. We consider real-valued time series and construct conditional distribution estimates that make much more efficient use of the input data. The estimates are consistent in a weak sense, and the question whether they are pointwise-consistent is still open. For finite-alphabet processes one may rely on a universal data compression scheme like the Lempel-Ziv (1978) algorithm to construct conditional probability mass function estimates that are consistent in expected information divergence. Consistency in this strong sense cannot be attained in a universal sense for all stationary processes with values in an infinite alphabet, but weak consistency can. Some applications of the estimates to on-line forecasting, regression, and classification are discussed