Communications of the ACM
Perceptrons: expanded edition
PP is closed under intersection
STOC '91 Proceedings of the twenty-third annual ACM symposium on Theory of computing
The expressive power of voting polynomials
STOC '91 Proceedings of the twenty-third annual ACM symposium on Theory of computing
Learning decision trees using the Fourier spectrum
STOC '91 Proceedings of the twenty-third annual ACM symposium on Theory of computing
Surveys in combinatorics, 1993
Efficient noise-tolerant learning from statistical queries
STOC '93 Proceedings of the twenty-fifth annual ACM symposium on Theory of computing
On the computational power of depth 2 circuits with threshold and modulo gates
STOC '94 Proceedings of the twenty-sixth annual ACM symposium on Theory of computing
Weakly learning DNF and characterizing statistical query learning using Fourier analysis
STOC '94 Proceedings of the twenty-sixth annual ACM symposium on Theory of computing
The harmonic sieve: a novel application of Fourier analysis to machine learning theory and practice
The harmonic sieve: a novel application of Fourier analysis to machine learning theory and practice
Learning Intersections and Thresholds of Halfspaces
FOCS '02 Proceedings of the 43rd Symposium on Foundations of Computer Science
New degree bounds for polynomial threshold functions
Proceedings of the thirty-fifth annual ACM symposium on Theory of computing
Noise-tolerant learning, the parity problem, and the statistical query model
Journal of the ACM (JACM)
A Random Sampling based Algorithm for Learning the Intersection of Half-spaces
FOCS '97 Proceedings of the 38th Annual Symposium on Foundations of Computer Science
Learnability and Automatizability
FOCS '04 Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science
Separating AC0 from depth-2 majority circuits
Proceedings of the thirty-ninth annual ACM symposium on Theory of computing
Evolvability from learning algorithms
STOC '08 Proceedings of the fortieth annual ACM symposium on Theory of computing
Refined experts: improving classification in large taxonomies
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Hi-index | 0.00 |
We prove new lower bounds for learning intersections of halfspaces, one of the most important concept classes in computational learning theory. Our main result is that any statistical-query algorithm for learning the intersection of $\sqrt{n}$halfspaces in n dimensions must make $2^{\Omega(\sqrt{n})}$queries. This is the first non-trivial lower bound on the statistical query dimension for this concept class (the previous best lower bound was n$^{\Omega(log{\it n})}$). Our lower bound holds even for intersections of low-weight halfspaces. In the latter case, it is nearly tight. We also show that the intersection of two majorities (low-weight halfspaces) cannot be computed by a polynomial threshold function (PTF) with fewer than n$^{\Omega((log{\it n})/loglog{\it n})}$monomials. This is the first super-polynomial lower bound on the PTF length of this concept class, and is nearly optimal. For intersections of k=ω(logn) low-weight halfspaces, we improve our lower bound to $\min\{2^{\Omega(\sqrt{n})},n^{\Omega(k/\log k)}\},$which too is nearly optimal. As a consequence, intersections of even two halfspaces are not computable by polynomial-weight PTFs, the most expressive class of functions known to be efficiently learnable via Jackson’s Harmonic Sieve algorithm. Finally, we report our progress on the weak learnability of intersections of halfspaces under the uniform distribution.