Probabilistic communication complexity
Journal of Computer and System Sciences
Lower Bound Methods and Separation Results for On-Line Learning Models
Machine Learning - Computational learning theory
Large Margin Classification Using the Perceptron Algorithm
Machine Learning - The Eleventh Annual Conference on computational Learning Theory
Improved Generalization Through Explicit Optimization of Margins
Machine Learning
An Algorithmic Theory of Learning: Robust Concepts and Random Projection
FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
A Linear Lower Bound on the Unbounded Error Probabilistic Communication Complexity
CCC '01 Proceedings of the 16th Annual Conference on Computational Complexity
Weighted decomposition kernels
ICML '05 Proceedings of the 22nd international conference on Machine learning
Theoretical Computer Science - Algorithmic learning theory(ALT 2002)
Computational Complexity
Learning complexity vs communication complexity
Combinatorics, Probability and Computing
VC dimension and inner product space induced by Bayesian networks
International Journal of Approximate Reasoning
Complexity Lower Bounds using Linear Algebra
Foundations and Trends® in Theoretical Computer Science
A lower bound for agnostically learning disjunctions
COLT'07 Proceedings of the 20th annual conference on Learning theory
SIAM Journal on Computing
On the properties of concept classes induced by multivalued Bayesian networks
Information Sciences: an International Journal
A little advice can be very helpful
Proceedings of the twenty-third annual ACM-SIAM symposium on Discrete Algorithms
On the limitations of embedding methods
COLT'05 Proceedings of the 18th annual conference on Learning Theory
COLT'05 Proceedings of the 18th annual conference on Learning Theory
COLT'05 Proceedings of the 18th annual conference on Learning Theory
Adaptive matching based kernels for labelled graphs
PAKDD'10 Proceedings of the 14th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part II
The approximate rank of a matrix and its algorithmic applications: approximate rank
Proceedings of the forty-fifth annual ACM symposium on Theory of computing
Hi-index | 0.00 |
The notion of embedding a class of dichotomies in a class of linear half spaces is central to the support vector machines paradigm. We examine the question of determining the minimal Euclidean dimension and the maximal margin that can be obtained when the embedded class has a finite VC dimension. We show that an overwhelming majority of the family of finite concept classes of any constant VC dimension cannot be embedded in low-dimensional half spaces. (In fact, we show that the Euclidean dimension must be almost as high as the size of the instance space.) We strengthen this result even further by showing that an overwhelming majority of the family of finite concept classes of any constant VC dimension cannot be embedded in half spaces (of arbitrarily high Euclidean dimension) with a large margin. (In fact, the margin cannot be substantially larger than the margin achieved by the trivial embedding.) Furthermore, these bounds are robust in the sense that allowing each image half space to err on a small fraction of the instances does not imply a significant weakening of these dimension and margin bounds. Our results indicate that any universal learning machine, which transforms data into the Euclidean space and then applies linear (or large margin) classification, cannot enjoy any meaningful generalization guarantees that are based on either VC dimension or margins considerations. This failure of generalization bounds applies even to classes for which "straight forward" empirical risk minimization does enjoy meaningful generalization guarantees.