Learning as Data Compression

Authors:
Pieter Adriaans
Affiliations:
Department of Computer Science, University of Amsterdam,Kruislaan 419, 1098VA Amsterdam, The Netherlands
Venue:
CiE '07 Proceedings of the 3rd conference on Computability in Europe: Computation and Logic in the Real World
Year:
2007

Citing 14
Cited 0

The minimum consistent DFA problem cannot be approximated within any polynomial

Journal of the ACM (JACM)
An introduction to Kolmogorov complexity and its applications (2nd ed.)

An introduction to Kolmogorov complexity and its applications (2nd ed.)
Machine Learning

Machine Learning
The Role of Occam‘s Razor in Knowledge Discovery

Data Mining and Knowledge Discovery
Information Compression by Multiple Alignment, Unification and Search as a Unifying Principle in Computing and Cognition

Artificial Intelligence Review
Results of the Abbadingo One DFA Learning Competition and a New Evidence-Driven State Merging Algorithm

ICGI '98 Proceedings of the 4th International Colloquium on Grammatical Inference
The EMILE 4.1 Grammar Induction Toolbox

ICGI '02 Proceedings of the 6th International Colloquium on Grammatical Inference: Algorithms and Applications
Unifying Computing and Cognition

Unifying Computing and Cognition
Suboptimal behavior of Bayes and MDL in classification under misspecification

Machine Learning
Using MDL for grammar induction

ICGI'06 Proceedings of the 8th international conference on Grammatical Inference: algorithms and applications
Learning deterministic DEC grammars is learning rational numbers

ICGI'06 Proceedings of the 8th international conference on Grammatical Inference: algorithms and applications
The minimum description length principle in coding and modeling

IEEE Transactions on Information Theory
Kolmogorov's structure functions and model selection

IEEE Transactions on Information Theory
Clustering by compression

IEEE Transactions on Information Theory

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper I describe the general principles of learning as data compression. I introduce two-part code optimization and analyze the theoretical background in terms of Kolmogorov complexity. The good news is that the optimal compression theoretically represents the optimal interpretation of the data, the bad news is that such an optimal compression cannot be computed and that an increase in compression not necessarily implies a better theory. I discuss the application of these insights to DFA induction.