Learnability and the Vapnik-Chervonenkis dimension
Journal of the ACM (JACM)
Journal of Computer and System Sciences
On ordinal VC-dimension and some notions of complexity
Theoretical Computer Science - Algorithmic learning theory
| Hi-index | 0.01 |
Vapnik and Chervonenkis proposed in [7] a combinatorial notion of dimension that reflects the ‘combinatorial complexity’ of families of sets. In the three decades that have passed since that paper, this notion – the Vapnik–Chervonenkis dimension (VC-dimension) – has been discovered to be of primal importance in quite a wide variety of topics in both pure mathematics and theoretical computer science.In this paper we turn our attention to classes with infinite VC-dimension, a realm thrown into one big bag by the usual VC-dimension analysis. We identify three levels of combinatorial complexity of classes with infinite VC-dimension. We show that these levels fall under the set-theoretic definition of σ-ideals (in particular, each of them is closed under countable unions), and that they are all distinct. The first of these levels (i.e., the family of ‘small’ infinite-dimensional classes) coincides with the family of classes which are non-uniformly PAC-learnable.Maybe the most surprising contribution of this work is the discovery of an intimate relation between the VC-dimension of a class of subsets of the natural numbers and the Lebesgue measure of the set of reals defined when these subsets are viewed as binary representations of real numbers.As a by-product, our investigation of the VC-dimension-induced ideals over the reals yields a new proper extension of the Lebesgue measure. Another offshoot of this work is a simple result in probability theory, showing that, given any sequence of pairwise independent events, any random event is eventually independent of the members of the sequence.