SODA '03 Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms
ISAAC '02 Proceedings of the 13th International Symposium on Algorithms and Computation
Journal of Logic, Language and Information
Optimality of universal Bayesian sequence prediction for general loss and alphabet
The Journal of Machine Learning Research
Concentration Theorems for Entropy and Free Energy
Problems of Information Transmission
MicroRNA target detection and analysis for genes related to breast cancer using MDLcompress
EURASIP Journal on Bioinformatics and Systems Biology
Algorithmic Minimal Sufficient Statistic Revisited
CiE '09 Proceedings of the 5th Conference on Computability in Europe: Mathematical Theory and Computational Practice
Kolmogorov Complexity and Model Selection
CSR '09 Proceedings of the Fourth International Computer Science Symposium in Russia on Computer Science - Theory and Applications
The computational structure of spike trains
Neural Computation
ICALP'03 Proceedings of the 30th international conference on Automata, languages and programming
Information measures for infinite sequences
Theoretical Computer Science
Biological information as set-based complexity
IEEE Transactions on Information Theory - Special issue on information theory in molecular biology and neuroscience
Rate distortion and denoising of individual data using Kolmogorov complexity
IEEE Transactions on Information Theory
Effective complexity and its relation to logical depth
IEEE Transactions on Information Theory
Causal inference using the algorithmic Markov condition
IEEE Transactions on Information Theory
An information theoretic representation of agent dynamics as set intersections
AGI'11 Proceedings of the 4th international conference on Artificial general intelligence
Non-approximability of the randomness deficiency function
CSR'06 Proceedings of the First international computer science conference on Theory and Applications
Replacing Causal Faithfulness with Algorithmic Independence of Conditionals
Minds and Machines
The digital universe - an information theoretical analyses
Proceedings of the 14th International Conference on Computer Systems and Technologies
Hi-index | 755.02 |
While Kolmogorov (1965, 1983) complexity is the accepted absolute measure of information content of an individual finite object, a similarly absolute notion is needed for the relation between an individual data sample and an individual model summarizing the information in the data, for example, a finite set (or probability distribution) where the data sample typically came from. The statistical theory based on such relations between individual objects can be called algorithmic statistics, in contrast to classical statistical theory that deals with relations between probabilistic ensembles. We develop the algorithmic theory of statistic, sufficient statistic, and minimal sufficient statistic. This theory is based on two-part codes consisting of the code for the statistic (the model summarizing the regularity, the meaningful information, in the data) and the model-to-data code. In contrast to the situation in probabilistic statistical theory, the algorithmic relation of (minimal) sufficiency is an absolute relation between the individual model and the individual data sample. We distinguish implicit and explicit descriptions of the models. We give characterizations of algorithmic (Kolmogorov) minimal sufficient statistic for all data samples for both description modes-in the explicit mode under some constraints. We also strengthen and elaborate on earlier results for the “Kolmogorov structure function” and “absolutely nonstochastic objects”-those objects for which the simplest models that summarize their relevant information (minimal sufficient statistics) are at least as complex as the objects themselves. We demonstrate a close relation between the probabilistic notions and the algorithmic ones: (i) in both cases there is an “information non-increase” law; (ii) it is shown that a function is a probabilistic sufficient statistic iff it is with high probability (in an appropriate sense) an algorithmic sufficient statistic